Skip to content

[WIP, RFC] drivers/eeprom: Common API to access EEPROM memory#12091

Closed
maribu wants to merge 13 commits intoRIOT-OS:masterfrom
maribu:eeprom_common_api
Closed

[WIP, RFC] drivers/eeprom: Common API to access EEPROM memory#12091
maribu wants to merge 13 commits intoRIOT-OS:masterfrom
maribu:eeprom_common_api

Conversation

@maribu
Copy link
Copy Markdown
Member

@maribu maribu commented Aug 26, 2019

Contribution description

This PR sketches how a generic EEPROM API could look like and be implemented. The user-facing API is mostly identical to the API in periph/eeprom.h in current master.

Testing procedure

In this early stage testing is not yet possible. Once the API has been agreed on, EEPROM drivers could be adapted to this API and then be tested.

Issues/PRs references

In PR #11929 a driver for external I2C EEPROM is suggested. The need for a common EEPROM API instead of a duct-tape wrapper over the existing EEPROM API became apparent to allow using external and internal EEPROMs with a consistent API

@maribu maribu added Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR Process: API change Integration Process: PR contains or issue proposes an API change. Should be handled with care. Area: drivers Area: Device drivers Process: needs >1 ACK Integration Process: This PR requires more than one ACK labels Aug 26, 2019
@maribu maribu requested review from aabadie and benpicco August 26, 2019 16:47
Copy link
Copy Markdown
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for trying to provide a unified API. It's always the simplest to discuss on existing code and you wrote it quickly. The proposed design is also more less the idea I had in mind.

Some thoughts/questions regarding the design:

  • Now one has to explicitly pass the device used if there are multiple EEPROM available. This changes quite a bit the existing API and will require user code to be updated.
  • To remove the need of the extra eeprom_t, how about only use one EEPROM at a time ? An external one being used instead of the CPU one if present ?
  • Another solution could be to "merge" all available EEPROM into a single global memory ?

@maribu
Copy link
Copy Markdown
Member Author

maribu commented Aug 27, 2019

* Now one has to explicitly pass the device used if there are multiple EEPROM available. This changes quite a bit the existing API and will require user code to be updated.

Like for every other RIOT-API. Citing the Guide for writing a device driver in RIOT:

Additionally towards ease of use, all device drivers in RIOT should provide a similar 'look and feel'.

  • To remove the need of the extra eeprom_t, how about only use one EEPROM at a time ? An external one being used instead of the CPU one if present ?

Citing again from the RIOT driver guide:

Third, it should always be possible, to handle more than a single device of one kind. Drivers and their interfaces are thus designed to keep their state information in a parameterized location instead of driver defined global variables.

So this is against RIOT's policy.

* Another solution could be to "merge" all available EEPROM into a single global memory ?

This is super complex. Image one has three EEPROM devices. Now the order of the initialization is changed --> the memory layout is messed up. Or one of the three fails to initialize. What will the memory layout look like? There more I think about it the more pitfall and corner cases pop up.

@aabadie
Copy link
Copy Markdown
Contributor

aabadie commented Aug 27, 2019

Citing the Guide for writing a device driver in RIOT

Make sense. I just wanted to discuss possible alternatives.
The current design makes it little more complex for the calling code which has to deal with available EEPROM source.

*/
static inline int eeprom_clear(eeprom_t dev, eeprom_off_t pos, size_t len)
{
return eeprom_set(dev, pos, 0, len);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some EEPROMs do provide pagewise erase.

benpicco
benpicco previously approved these changes Aug 30, 2019
Copy link
Copy Markdown
Contributor

@benpicco benpicco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good for a basic EEPROM interface.
It behaves just like the other driver subsystems and if we want to add some optimizations for special EEPROMs, we can still do that later.

* @retval 0 Failed to initialize the device
* @return The size of the device in bytes
*/
eeprom_off_t (*init)(void *handle, const void *params);
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aabadie: You suggested to drop the params here, right?

const eeprom_driver_t *driver; /**< The driver of the EEPROM device */
const void *params; /**< The initialization parameters */
void *handle; /**< The handle to pass to the driver */
} eeprom_dev_t;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aabadie: I think if we drop the params in the init callback, it will no longer be possible to add multiple external EEPROM devices that are using the same driver, without some dark magic. Assume the init function of an external driver is only called with the device handle to initialize. With what parameters should it initialize that device handle?

If there is only one device of that kind, it is clear. But RIOT policy is to allow multiple devices of a kind. I don't see how this could work.

@benpicco benpicco requested review from dylad and kaspar030 September 16, 2019 22:14
@benpicco
Copy link
Copy Markdown
Contributor

What's the verdict on this one?

@benpicco benpicco requested a review from fjmolinas October 14, 2019 11:58
@benpicco
Copy link
Copy Markdown
Contributor

I'm fine with this API.
If the concern is that this adds too much overhead/unneeded indirection because nobody will ever use more than one EEPROM at a time, I can also draft a header-only api (with some common support code) like what we have for the watchdog.

@benpicco benpicco requested a review from a team October 28, 2019 10:40
@benpicco benpicco requested a review from aabadie November 6, 2019 01:46
@benpicco
Copy link
Copy Markdown
Contributor

benpicco commented Nov 6, 2019

I also provide a PR as fall back wrapper for the existing API, that simple uses the first EEPROM device available - that way external applications could work without being updated.

This could also include a fast path to directly (or with a thin wrapper) call the periph_eeprom functions if no external EEPROM modules are used.

void eeprom_init(void)
{
for (unsigned i = 0; i < EEPROM_NUMOF; i++) {
_sizes[i] = eeprom_devs[i].driver->init(eeprom_devs[i].handle,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use

        assert(eeprom_devs[i].driver && eeprom_devs[i].handle)

here and in all other functions that use these pointers to avoid silent crashes.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. I just thought that some drivers might not want an handle. E.g. for a periph device, the number of device is fixed. Maybe that driver prefers to have the handle as NULL and handle the device state locally, as it is safe to statically allocate a fixed amount of memory for that specific case.

But the assert() for the driver is a very good idea. Thanks!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. I just thought that some drivers might not want an handle.

OK, that's a good argument. If the driver insists on a valid handle, the driver has to use assert for the handle.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should decouple underlying device params from the common eeprom init function. See how this is done with netdev.
I would also use an eeprom_t dev parameter for the init function.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So providing a setup() taking the parameters that and an init() function? I never fully understood why that was done, as it increase ROM size (both by having two functions to call, and by having two times function boilerplate instructions needed to comply with the caller convention), increases the complexity (now two functions have to be called instead of one), increases the sources for errors (the order of the function calls matters), and has no apparent benefit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @aabadie. The EEPROM API should be free from device specific stuff. That would also imply that the API only defines the type eeprom_dev_t for the device list eeprom_devs. The device list eeprom_devs would be defined by the board definition.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about users that use external EEPROM, that is not part of the board?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about moving the initialization out of this API and providing a function to register an eeprom device, similar to the SAUL registry. The initialization could than be performed in auto init with a single init function for each device. It would still not overcome the issue that users have to count the number of EEPROM devices, but it would provide the strict separation of configuration parameters and common API without the overhead of two init functions per device.

Copy link
Copy Markdown
Contributor

@gschorcht gschorcht Nov 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about users that use external EEPROM, that is not part of the board?

They could define such a configuration file in $(APPDIR). According to the example in PR #12333 #10688 #10430 there would be a file eeprom_dev_conf.h which is included by the eeprom module. This file would be usualy in board definition but could also be in $APPDIR provided that that $(APPDIR) the first directory in the INCLUDES path.

EEPROM_NUMOF would also be evaluated during compile time.

Copy link
Copy Markdown
Contributor

@gschorcht gschorcht Nov 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

provided that that $(APPDIR) the first directory in the INCLUDES path.

BTW, at some point I suggested adding $(APPDIR) as the first entry in INCLUDES by default to be able to override the default board definition by the application. But that didn't find consensus. Thus the application has to do it explicitly if required.

@maribu
Copy link
Copy Markdown
Member Author

maribu commented Nov 8, 2019

OK. I split out initialization of this PR entirely and added a function void eeprom_register(const eeprom_driver_t *driver, void *handle, eeprom_off_t size); to registers devices to the common API instead. I think this could be a good compromise.

@maribu maribu dismissed benpicco’s stale review November 8, 2019 12:54

PR has changed significantly since the ACK. A new ACK is needed.

Copy link
Copy Markdown
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the current approach. Maybe that would be great to implement some unittest of the new common API ? I think there's no need for using the existing periph_eeprom driver, just use a fake EEPROM in RAM or something.
This ensure compilation of the current approach it not broken and the whole concept is functional. What do you think ?

Otherwise I have other inline comments. The main one is about the use of some static inline functions.

#endif

/* Include implementation of the static inline functions */
#include "eeprom_implementation.h"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a real benefit with the static inline functions ? Is not I would just implement declare them here and implement them in eeprom.c. Or maybe keep them static inline but put them in eeprom.h.

It's just that I don't like having to include a header at the end of another one so it's better if we could avoid this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a real benefit with the static inline functions?

It depends. In any case, it will be faster. In some cases, the compiler will be able to optimize out (some of) the range checks done there.

Regarding ROM size, this could be both a minor improvement or a minor regression. It will depend on the number of places where the common API is called and on how much of the code can be optimized out. In general, I expect small applications to be slightly small with this, and other being slightly bigger.

I have no strong feelings about this; so I'd just as happy move it back to eeprom.c.

@benpicco suggested to reduce the overhead here, maybe he wants to comment before I move that back?

It's just that I don't like having to include a header at the end of another one so it's better if we could avoid this.

This could actually also be included at the top of the file, if no compiler flags are set to warn about late declarations. But I think the C standard is fine with multiple or late declarations (as long as declarations and the implementations match in name and signature). The advantage of still having the declaration here is that the generated documentation will be properly grouped. And the advantage of moving the implementation out is the separation of API and implementation.

This is by the way also not uncommon, e.g. xtimer.h includes the static inline implementations at the end of the header.

@gschorcht
Copy link
Copy Markdown
Contributor

gschorcht commented Nov 9, 2019

I like the current approach.

I still don't really like the approach of registration. Usually in RIOT all hardware components are defined by static initializers. Why not use the same for the configuration of existing EEPROM devices? If _eeprom_devs[] would be declared by the board or the application like following

static const eeprom_dev_t _eeprom_devs[] = {
    {
      .driver = abc_eeprom_driver,
      .handle = &abc_eeprom,
      .params = &abc_eeprom_params,
      .size = ABC_EEPROM_SIZE
    },
    {
      .driver = xyz_eeprom_driver,
      .handle = &xyz_eeprom,
      .params = &xyz_eeprom_params,
      .size = XYZ_EEPROM_SIZE
    },
};

the init function of the API could iterate over all EEPROM devices and could call

    _eeprom_driver(dev)->init(_eeprom_handle(dev), _eeprom_params(dev));

The init function could be called in sys/auto_init.

This approach would have the advantages

  • that initialization of all EEPROMs would be done automatically
  • the application has not take care what functions to call and when
  • the EEPROM_NUM_OF would be simply `ARRAY_SIZE(_eeprom_devs)

@benpicco
Copy link
Copy Markdown
Contributor

FYI: Some EEPROMs provide an EUI-48 or EUI-64.

@maribu
Copy link
Copy Markdown
Member Author

maribu commented Nov 16, 2019

I still don't really like the approach of registration.

Well, this is basically the trade-off we have to make. Either we pull in implementation details of every implementation to allow for static initialization, or we keep the API completely distinct from those details. Recently, you and @aabadie convinced me that the latter is to be preferred in order to trade initialization overhead for better maintainability and sound architectural design.

Usually in RIOT all hardware components are defined by static initializers.

E.g. with SAUL and network device drivers we have the registration for the same reason.

If _eeprom_devs[] would be declared by the board or the application

This would have its own disadvantage; namely that board configurations get longer and that end users plugging external stuff (not included in their board) would have more effort to configure.

I have however no strong feeling about this decision. Maybe @aabadie and @benpicco could give their opinion on whether they prefer registration or static initializers? I would than just do as the majority says. (Luckily there are three of you, so there should be no draw 😄)

@benpicco
Copy link
Copy Markdown
Contributor

If the registration approach helps keep the board files cleaner I'm all for it.
And if we manage to resist the temptation to compare EEPROM_NUMOF in processor macros we can even populate it automatically:

#define EEPROM_NUMOF (MCU_EEPROM + AT24XXX_NUMOF + AT25XXX_NUMOF …)

@benpicco
Copy link
Copy Markdown
Contributor

@gschorcht What about EEPROMs that are integrated into the MCU? It would be quite bad if every board would have to declare them.

@gschorcht
Copy link
Copy Markdown
Contributor

gschorcht commented Nov 28, 2019

@gschorcht What about EEPROMs that are integrated into the MCU? It would be quite bad if every board would have to declare them.

For boards that have same properties, we use boards/common/*/board_common.h for the definitions.

But that's not my point. All hardware configurations in RIOT are done as static const definitions during the compile time. There is only one exception, the netif. However, we are still looking for an approach on how to get the static definition of GNRC_NETIF_NUMOF and the dynamic netif instantiation consistent.

This will be the same here. We have the static definition of EEPROM_NUMF and the dynamic registration of the devices that are really there. Also your macro hack wouldn't solve this problem since it doesn't cover the case that a board could have multiple eeproms of the same type.

The attempt of a dynamic instantiation without having a real dynamic registry just has its limits 😟

To be clear, I'm not against the approach. It just wanted to tell my concerns. There are a lot of other use case where such a registration would help, e.g., for GPIO extenders.

@benpicco
Copy link
Copy Markdown
Contributor

benpicco commented Nov 28, 2019

For boards that have same properties, we use boards/common/*/board_common.h for the definitions.

That doesn't help with your static array example.
Say you have an ATmega board with an additional I2C EEPROM.
You can't have a split array between common code and the board, so either you have to declare both EEPROMs in the board code (meh) or you do something like

#define EXTERNAL_EEPROMS            \
    {                               \
      .driver = i2c_eeprom_driver,  \
      .params = &i2c_eeprom_params, \ 
      .size = I2C_EEPROM_SIZE       \
    },

…

#define INTERNAL_EEPROMS                \
    {                                   \
      .driver = atmega_eeprom_driver,   \
      .params = NULL,                   \
      .size = ATMEGA_EEPROM_SIZE        \
    },

…

static const eeprom_dev_t _eeprom_devs[] = {
    INTERNAL_EEPROMS 
    EXTERNAL_EEPROMS 
};

Or just have a separate struct for external EEPROMs - that should work too.

But on top of that you also need

#define AT24XXXX_PARAMS         {           \
        .i2c = AT24XXXX_PARAM_I2C,          \
        .dev_addr = AT24XXXX_PARAM_ADDR,    \
        .pin_wp = AT24XXXX_PARAM_ADDR,      \
        .eeprom_size = AT24XXXX_PARAM_EEPROM_SIZE \
}

for the driver itself.

Also your macro hack wouldn't solve this problem since it doesn't cover the case that a board could have multiple eeproms of same type.

If you have

#define AT24XXX_NUMOF ARRAY_SIZE(at25xxx_params)

it should work.

@benpicco
Copy link
Copy Markdown
Contributor

benpicco commented Mar 22, 2020

I'm wondering: Why not just use the existing MTD API for EEPROMs?

@maribu
Copy link
Copy Markdown
Member Author

maribu commented Mar 25, 2020

Any idea why MTD exposes the concept of sectors and pages? The API seems not to be using this. IMHO it would be nice to get rid of these details, so that MTD could be applied more broadly. But to me it seems that indeed that API should work just fine.

Some work would be needed to update users like the EEPROM Registry, so that they could be used with external EEPROM. (And any other memory that has an MTD API.)

Btw: Wouldn't storage would be the correct term rather than memory? To me, storage is persistent, while memory is volatile.

@maribu
Copy link
Copy Markdown
Member Author

maribu commented Apr 23, 2020

Just using the MTD API is clearly the better approach. No need to reinvent the wheel.

@maribu maribu closed this Apr 23, 2020
@maribu maribu deleted the eeprom_common_api branch May 11, 2020 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: drivers Area: Device drivers Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR Process: API change Integration Process: PR contains or issue proposes an API change. Should be handled with care. Process: needs >1 ACK Integration Process: This PR requires more than one ACK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants