Add MCUBoot linking options for RIOT images#7209
Conversation
cpu/nrf52/ldscripts/nrf52832xxaa.ld
Outdated
| .imghdr (NOLOAD): | ||
| { | ||
| . = . + img_hdr; | ||
| } > rom |
There was a problem hiding this comment.
I think this part should be added to cortexm_base.ld
What do you think @kaspar030 ?
There was a problem hiding this comment.
what is it used for? is it even necessary?
There was a problem hiding this comment.
It represents the image headers (version, size, etc.) which are generated by imgtool. The goal to have them here is to generate the necessary "offset" indicated by img_hdr which ease the manipulation by objcopy and other tools trying to read the ELF file.
There was a problem hiding this comment.
The goal to have them here is to generate the necessary "offset" indicated by img_hdr which ease the manipulation by objcopy and other tools trying to read the ELF file.
Is there any use other than the offset?
If not, specifying it in the linkerscript AND adding the actual size as parameter to the linking call is not necessary. Just adjust the linking offset (add 512). If the actual position of the metadata cannot be deduced by "image start - sizeof(metadata)", define either it's position or its offset via linker parameter.
My point is, there's no need to modify the linker script for this. Thus we should not do it, to keep maintainability as high as possible.
There was a problem hiding this comment.
I think keeping this variable in the linker script doesn't harm, and adds readability when looking at the ELF file where we can find the specific allocated space for the headers. This can be useful when using different tools to add these headers directly to the ELF file, instead if padding them to the binary file. It also helps to know while debugging where the real image starts, while also knowing why the vector table is located 512B after the bootloader.
There was a problem hiding this comment.
I think keeping this variable in the linker script doesn't harm
It needs adaption of the linker scripts
and adds readability when looking at the ELF file where we can find the specific allocated space for the headers.
true, but ...
This can be useful when using different tools to add these headers directly to the ELF file, instead if padding them to the binary file.
This is my most concern. If the header is part of the image, the complexity of signing images increases a lot. How's the signature generated? Is the header part of it? We'd also basically force usage of an ELF aware image signing tool, instead of simply cat'ing header and elf together.
It also helps to know while debugging where the real image starts, while also knowing why the vector table is located 512B after the bootloader.
- the debugger knows the image start regardless of the image hdr defined in the linker address
- the debugger doesn't mention the image header anywhere, regardless of it being defined in the linker script or not
- if the image is compiled with knowledge about MCUBOOT and it's headers, it can deduce the image hdr start, and knows it's length
I'm not convinced. The only reason to put this into the linker script is to give the linker a hint about the size of the image header, which can also be implicitly given by setting a correct offset. On the other hand, this adds to maintanance and complexity. so please remove.
There was a problem hiding this comment.
OK I understood your point, will change asap.
This is my most concern. If the header is part of the image, the complexity of signing images increases a lot. How's the signature generated? Is the header part of it? We'd also basically force usage of an ELF aware image signing tool, instead of simply cat'ing header and elf together.
This is different on MCUBoot, on which the signatures and hashes are part of a TLV, at the end of the image, so no problem at this point. However, if we want to reuse this code for the RIOT bootloader then yes, this is a big concern (and I want to reuse this of course).
the debugger knows the image start regardless of the image hdr defined in the linker address
This depends on how you start the debugger. In my very short experience, what I do is to first launch the debugger for the bootloader, as this is anyways the first piece of code to be executed. Then you need to know where the .text section for the next ELF file begins, in order to load it into the right address, which is not automatically read by the debugger. By having explicitly indicated the image headers, I was aware on which part the program begins, and so load the right address. Of course, this can be deduced from the text part, but you could wonder why the program starts at a "strange" address. Anyways, we can certainly live without such information.
the debugger doesn't mention the image header anywhere, regardless of it being defined in the linker script or not
Completely agree, but it was just for the case I explained above.
|
Ping @kaspar030 . As @utzig proposed, I'll offer you a beer for this review ;) |
|
There is a JIRA ticket tracking the progress of this PR, so once this is merged the issue can be marked as done. https://runtimeco.atlassian.net/projects/MCUB/issues/MCUB-61 |
|
@kaspar030 feature freeze is tomorrow! 😖 |
kaspar030
left a comment
There was a problem hiding this comment.
Minor things! Almost there.
cpu/nrf52/ldscripts/nrf52832xxaa.ld
Outdated
| .imghdr (NOLOAD): | ||
| { | ||
| . = . + img_hdr; | ||
| } > rom |
There was a problem hiding this comment.
what is it used for? is it even necessary?
makefiles/multislot.mk
Outdated
| $(error Board $(BOARD) does not define multislot parameters!) | ||
| endif | ||
|
|
||
| ifeq (nrf52dk, $(BOARD)) |
There was a problem hiding this comment.
please introduce a generally usable variable for this. This must be doable without board specifics int this file.
I'd suggest "FLASH_OFFSET" or sth similar, which would need to be exported, and used by our flashers (jlink, edbg, openocd)
tests/mcuboot/README.md
Outdated
| system. | ||
|
|
||
| This test should be called using `make mcuboot` to produce such ELF file, | ||
| which can also be flashed using `make mcuboot flash-mcuboot`. |
There was a problem hiding this comment.
flash-mcuboot depends on mcuboot, so specifying both is redundant
|
|
||
| BOARD_WHITELIST := nrf52dk | ||
|
|
||
| export IMAGE_VERSION = 1.1.1+1 |
There was a problem hiding this comment.
Why are the next two needed here?
There was a problem hiding this comment.
This variable is needed for imgtool, which only allows this kind of versioning format.
There was a problem hiding this comment.
I was asking about the following two. IMO they need to go into multiboot.mk, as they're not application dependent, but MCUBOOT dependent.
There was a problem hiding this comment.
Oh, those variables are CPU dependent, since the alignment can change according to the specific alignment of the internal flash (mostly is 4, but for some CPUs is 8). IMG_HDR is also CPU specific, so maybe I can move these two exports to the Makefile.include of the CPU.
There was a problem hiding this comment.
Moved to cpu/nrf52/Makefile.include
There was a problem hiding this comment.
- isn't the image always aligned on a flash page boundary?
- isn't the IMG_HDR always 512 for mcuboot?
There was a problem hiding this comment.
isn't the image always aligned on a flash page boundary?
Actually they define "alignment" not for the pages but for the minimum size they can write on the flash, maybe @utzig knows more about this, but afaik it can change. If we want to align to our flash page boundary it would be FLASHPAGE_SIZE which is a lot, and can't be read by imgtool.
isn't the IMG_HDR always 512 for mcuboot?
Not always afaik, and can be changed as needed. In mynewt is actually 32B, since they relocate the VTOR on RAM so they don't need a 512B alignment as it is needed if we locate the VTOR in ROM. However in RIOT I don't know if we aim to locate VTOR in RAM one day...
| @@ -0,0 +1,23 @@ | |||
| /* | |||
There was a problem hiding this comment.
copyright and author need updating
|
@kaspar030 can you see if the fixes are ok? |
makefiles/multislot.mk
Outdated
|
|
||
| ifndef SLOT0_SIZE | ||
| $(error Board $(BOARD) does not define multislot parameters!) | ||
| else |
There was a problem hiding this comment.
drop else and endif, "error" will break the build anyways.
makefiles/multislot.mk
Outdated
| @$(COLOR_ECHO) | ||
| @$(COLOR_ECHO) '${COLOR_PURPLE}Re-linking for MCUBoot at $(SLOT0_SIZE)...${COLOR_RESET}' | ||
| @$(COLOR_ECHO) | ||
| $(Q)$(_LINK) $(LINKFLAGPREFIX)--defsym=offset="$(SLOT0_SIZE)" \ |
There was a problem hiding this comment.
here you can use --defsym=offset=$$(($(SLOT0_SIZE) + $(IMAGE_HDR))" and --defsym=length="$$(($(SLOT1_SIZE) - $(IMAGE_HDR)". IMAGE_HDR should also be renamed to IMAGE_HDR_SIZE.
--defsym=image_header... can then be dropped, along with the linker script addition for img_hdr.
There was a problem hiding this comment.
I just renamed the variable to IMAGE_HDR_SIZE.
tests/mcuboot/Makefile
Outdated
|
|
||
| export IMAGE_VERSION = 1.1.1+1 | ||
|
|
||
| default: mcuboot |
There was a problem hiding this comment.
@kaspar030 I still have the doubt on how we will trigger the correct build for this test.
|
please add a note that the python 3 packages "pyasn1", "pycrypto" and "ecdsa" need to be installed. |
|
Oh yeah, sorry, I made the same comment to the upstream repository and forgot a small readme on this one. Will do asap. |
|
@kaspar030 I added your suggestions we discussed offline. I think is ready for a final review. |
|
Doc added too. |
tests/mcuboot/README.md
Outdated
| This test should be called using `make mcuboot` to produce such ELF file, | ||
| which can also be flashed using `make flash-mcuboot`. | ||
|
|
||
| In order to get the program to run, it is necessary to have MCUBoot flashed |
There was a problem hiding this comment.
Oops, right, will fix now.
makefiles/multislot.mk
Outdated
| mcuboot: | ||
| $(Q)echo "error: mcuboot not supported on board $(BOARD)!" | ||
| $(Q)false | ||
| endif # mcuboot |
There was a problem hiding this comment.
should be endif # SLOT0_SIZE?!
There was a problem hiding this comment.
Opps, yes, it came from Kaspar's code but I didn't check. Will fix now.
cpu/nrf52/Makefile.include
Outdated
|
|
||
| # Export internal ROM alignment and image header size for bootloader support | ||
| export IMAGE_ALIGN = 8 | ||
| export IMAGE_HDR_SIZE = 512 |
There was a problem hiding this comment.
I wonder if these values are specific to the CPU (and if, I guess they should go into nrf5x_common...), or if they are the same for all cortex-m base CPUs?
There was a problem hiding this comment.
Yes, they're CPU dependent, at least the IMAGE_ALIGN variable. I think at the beginning I had them in the cpu common folder but for some reason I moved them here. I can move them again to the CPU common, however I'm not sure if IMAGE_HDR_SIZE depends on something. This variable is used by imgtool to add the right offset (512B in this case) at the beginning of the image, and it's always either 256B or 512B aligned, regardless of the headers size. It means however that if for some reason more than 512B of headers is needed, then this variable must be defined for 1024B, therefore, I think we cannot assume it's constant for all situations.
Besides, as I said in a previous comment, there's the possibility to relocate the VTOR in RAM, which allows to define the headers size to an "arbitrary" value (32B in the case of mynewt which locates VTOR on RAM). The 512B alignment is only needed when locating the VTOR on ROM, so the PC can jump to the defined address.
There was a problem hiding this comment.
I understand, that the IMAGE_HDR_SIZE can have different values, but these don't have anything to do with the board, but are dependent on CPU configuration. And as RIOT does not support putting the VTOR into RAM (yet), we don't need to worry about it. So I think this variable should be defined with a default value somewhere in the cpu/cortexm_common path instead of in the board path...
There was a problem hiding this comment.
OK, I agree, I didn't find the right place for such variable but we can put it there if that's the most convenient.
boards/nrf52dk/Makefile.include
Outdated
| export LINKER_SCRIPT ?= $(RIOTCPU)/$(CPU)/ldscripts/$(CPU_MODEL)_sd.ld | ||
| endif | ||
|
|
||
| export ROM_SIZE = 0x80000 |
There was a problem hiding this comment.
this value is never used?!
There was a problem hiding this comment.
Not yet, but it can be useful to define slot sizes depending on the ROM size. I can remove it if necessary.
There was a problem hiding this comment.
Then add it once it is actually used :-)
boards/nrf52dk/Makefile.include
Outdated
| export ROM_SIZE = 0x80000 | ||
| export SLOT0_SIZE = 0x8000 | ||
| export SLOT1_SIZE = 0x3C000 | ||
| export SLOT2_SIZE = 0x3C000 |
There was a problem hiding this comment.
and are these value really board-dependent? Aren't they rather something the CPU defines?
There was a problem hiding this comment.
I think I got confused on which variables I should put on either board or cpu code, but now I think everything is board-independent, so I can move this values safely.
There was a problem hiding this comment.
yes, please do. I think for we could put them into the nrf5x_common path and ifdef them with the actual CPU_MODELs. Or even better: why not introduce a global variable that defines the ROM size (defined in the CPU tree), an then do the slot allocation somewhere centrally?
There was a problem hiding this comment.
I think it helps on readability and better understanding on the partitioning if we set explicitly the slot sizes for the concerned CPU in the local CPU Makefile. Putting them or setting them in a place which can be hard to find would make difficult to find "why my code doesn't fit in ROM? Where the slots are being defined?". Does it harm on code size or something more critical?
| * @} | ||
| */ | ||
|
|
||
| rom_length = 512K; |
There was a problem hiding this comment.
This is not quite correct, I think the value of rom_length should be dependent on the boot_offset -> rom_length = 512K - boot_offset or even better: ROM length should be dependent on the slot chosen and on the slot size...
There was a problem hiding this comment.
This variable will be redefined if the multislot strategy is used, you can look at this line where the variable is modified when an offset is defined. Otherwise it stays with the default value.
There was a problem hiding this comment.
sorry, overlooked that one, so never mind.
dist/tools/jlink/jlink.sh
Outdated
| echo "Flashing at offset ${FLASH_OFFSET}" | ||
| else | ||
| _FLASH_ADDR=0 | ||
| fi |
There was a problem hiding this comment.
this seems unclean, you override the default flash address variable just for it to be checked again below... So now we use 2 different 'external' variables (FLASH_OFFSET and JLINK_FLASH_ADDR) do determine the actual used flash address... Even worse, if JLINK_FLASH_ADDR is defined by the CPU, then the FLASH_OFFSET variable will be ignored!
So this needs to be cleaned up.
There was a problem hiding this comment.
I also thought I could use the same variable, but when the time for new cpu support will come (very near future) we will anyways need a variable of this kind (e.g. for openocd, edbg, bossa, etc.).
There was a problem hiding this comment.
One more reason to 'globalize' this varialbe, of in other words to introduce a var called FLASH_ADDR that use used throughout all CPUs (possible with a default value set to 0) and get rid of the JLINK_FLASH_ADDR variable alltogether
There was a problem hiding this comment.
The drawback of that approach is that other flashing tools use different scripts to write into the flash from different sources. In this case the used variable JLINK_FLASH_ADDR is specific to JLink since it flashes a binary file, which has no information about "where to flash it". This is not the same case for openocd for instance, which is intended to flash an ELF or an HEX file (at least in our current scripts), without any extra address information (it's self contained). So I think due to these differences a "super global" approach seems to me complicated to implement without big changes on all the scripts.
There was a problem hiding this comment.
Just to complete this... JLINK_FLASH_ADDR is used only for the Nordic's BLE softdevice, which doesn't work since a longtime... So removing that variable AND the broken support for softdevice would be cleaner.
There was a problem hiding this comment.
if this var is really needed by the softdevice (and I think it is rather the softdevice adaption code that needs it...), it might as well be defined only by the softdevice specific code with a simple
JLINK_FLASH_ADDR = FLASH_ADDR or something...
There was a problem hiding this comment.
So I think due to these differences a "super global" approach seems to me complicated to implement without big changes on all the scripts
I disagree: having this 'standard' global variable does not hurt in any case. If the flash tool does not need it, simply don't use it. If there is something (whatever that my be) that needs to know the starting address of the flash, we have a defined way to define/retrieve it.
There was a problem hiding this comment.
if this var is really needed by the softdevice (and I think it is rather the softdevice adaption code that needs it...), it might as well be defined only by the softdevice specific code with a simple
The softdevice binary import instructions modifies this variable (which is only used by jlink.sh) to flash the RIOT code in a specific part of the ROM, but there's no other code in RIOT which needs to change this variable or makes use of it.
I disagree: having this 'standard' global variable does not hurt in any case. If the flash tool does not need it, simply don't use it. If there is something (whatever that my be) that needs to know the starting address of the flash, we have a defined way to define/retrieve it.
An example of what it needs to be done for other scripts is given in #7181, specifically here: 2aa95a7 where I do the modifications needed to use a "global" variable. Of course I agree to use a "standard" (if we can call it like that) global variable, I was only talking about the modifications needed to remove the current one to replace it by this "global" one (namely change the Makefile for the softdevice, very simple) which, IMHO, is out of the scope of this PR.
Anyways, if it's ok for you to do that change in this PR I can of course do it.
|
Comments addressed. |
|
|
||
| DIRS += $(EXTERNAL_MODULE_DIRS) | ||
|
|
||
| _LINK = $(if $(CPPMIX),$(CXX),$(LINK)) $(UNDEF) $(LINKFLAGPREFIX)--start-group $(BASELIBS) -lm $(LINKFLAGPREFIX)--end-group $(LINKFLAGPREFIX)-Map=$(BINDIR)/$(APPLICATION).map $(LINKFLAGPREFIX)--cref $(LINKFLAGS) |
There was a problem hiding this comment.
b.t.w. whats up with the underscore here? We are not using this notation anywhere else in this file...
There was a problem hiding this comment.
That was Kaspar's idea, it looks also a bit strange for me but it's just maybe a way to prevent the variable being used or modified.
There was a problem hiding this comment.
hmm, I don't like it, but don't really care too much about it. Just wanted to make sure it was intentional :-)
There was a problem hiding this comment.
Yeah well... actually there's the variable __DIRECTORIES or something like that more up in this file, so this one is not the only one...
There was a problem hiding this comment.
then at least the use of underscores should be consistent :-) But leave it...
| export LINKER_SCRIPT ?= $(RIOTCPU)/$(CPU)/ldscripts/$(CPU_MODEL)_sd.ld | ||
| endif | ||
|
|
||
| include $(RIOTMAKE)/tools/jlink.inc.mk |
| export IMAGE_ALIGN = 8 | ||
| export SLOT0_SIZE = 0x8000 | ||
| export SLOT1_SIZE = 0x3C000 | ||
| export SLOT2_SIZE = 0x3C000 |
There was a problem hiding this comment.
didn't we talk about this, that these values are not board specific?
There was a problem hiding this comment.
But I would be o.k. to postpone the fix of this to a follow up PR and search for a more generic solution for 'slicing' the ROM.
There was a problem hiding this comment.
Well, I thought cpu/nrf52/Makefile.include belongs to cpu code and not to board code... Then I don't know where to put this variables (which IMHO belong only to this specific model of CPU).
There was a problem hiding this comment.
sorry, my mistake, somehow I read nrf52dk... Never mind, seems I see things that are not there today...
There was a problem hiding this comment.
ideally, we define rom_size, bootloafer siue anf flashpage size, and the three slot sizes get calculated from that. we'll probavly support multiple bootloaders, soon.
There was a problem hiding this comment.
Yeah that was the idea behind the definition of the size, which is removed now but I'll add it in the next PR.
There was a problem hiding this comment.
Agree with @kaspar030 (and that is what I had in mind in some earlier comments somewhere), but let's merge this for now and clean this up later.
By the way, this unconditional slot size definition is also calling for trouble as there are nrf52s with differing ROM size out there... But lets fix later.
makefiles/vars.inc.mk
Outdated
| export SIZE # The command to read to size of the ELF sections. | ||
| export UNDEF # Set by the BOARD's and CPU's Makefile.include, this contains object files with must not be used in the ELFFILE even if the if no call to the functions. | ||
| export WERROR # Treat all compiler warnings as errors if set to 1 (see -Werror flag in GCC manual) | ||
| export FLASH_OFFSET # Define an offset to flash code into ROM memory. |
There was a problem hiding this comment.
@haukepetersen BTW I think this can be the beginning of the definition for such a global variable, the changes to the code which needs it can come one after the other.
There was a problem hiding this comment.
Actually this might be a good place. My suggestion would be, to insert a section for setting default values on top of this file, as in
# set default values for selected global variables
FLASH_ADDR ?= 0
# export global variables defined by the make system
export Q # Used in front of Makefile lines to suppress the printing of the command if user did not opt-in to see them.
export QQ # as Q, but be more quiet
...Further I would prefer to (i) name the varialbe FLASH_ADDR and second to put the export line it it's own block (as this define is not tool related but CPU related)
There was a problem hiding this comment.
Ok great! I like this. So you propose to export the variable in makefiles/cortexm.inc.mk?
There was a problem hiding this comment.
Oh ok I think I misunderstood. You want to keep the variable in the same file but put it in a different block right? And if that's also for the value setting I don't think that's the right place, since there's no variable set in that file, but only exported.
There was a problem hiding this comment.
you mean 'not yet' :-) Why not start with this variable and see if there are others that we can assign a default fallback value here as well?!
There was a problem hiding this comment.
Well looking at the history of that file I have the impression it was not meant to set variables there, but it seems it was intentionally created for only the purpose of exporting, maybe I'm wrong...
There was a problem hiding this comment.
But well, of course if the change makes sense, we can start to do that if it's useful, it's just that my impression is that maybe we're creating a "blackbox" since some variables can be set there by default, thus some things outside this file will work "like magic".
There was a problem hiding this comment.
Well looking at the history of that file I have the impression it was not meant to set variables there, but it seems it was intentionally created for only the purpose of exporting, maybe I'm wrong...
no, it actually was created for that reason, but I also don't see a reason for extending its purpose. There is actually no better place to set global default fallback values...
Well looking at the history of that file I have the impression it was not meant to set variables there, but it seems it was intentionally created for only the purpose of exporting, maybe I'm wrong...
Not quite true: we do default values for many variables, its just that we do it all over the place currently... So this would IMHO even be a step towards bringing more structure into the existing blackbox...
There was a problem hiding this comment.
OK, I'll do the change then.
|
@haukepetersen better now? |
haukepetersen
left a comment
There was a problem hiding this comment.
One minor thing, otherwise I would say we can go ahead with this PR (if you promise to advance with the slotsize computation next...)
| fi | ||
| if [ -z "${JLINK_FLASH_ADDR}" ]; then | ||
| JLINK_FLASH_ADDR=${_FLASH_ADDR} | ||
| fi |
There was a problem hiding this comment.
I would prefer that we still check, if FLASH_ADDR is set before proceeding, as this script could be called from outside the RIOT makesystem... So a simple if [ -z "${FLASH_ADDR}" ]; then -> echo error message should do...
|
I guess now it's everything done. Ok to squash? |
haukepetersen
left a comment
There was a problem hiding this comment.
ACK, go ahead and squash
|
Squashed, waiting for Murdock. |
|
Murdock is ok. @kaspar030 are you ok with the changes? |
|
@kaspar030 is on holiday and will be back in ~2 weeks... As I see it, hier comments are addressed. If not, we fix it as soon as he is back. So I take it on me to dismiss his review... |
As far as I see it, the comments are addressed.
|
Well, let's merge then! |
|
just in time for the MCUboot release? |
|
Yes! It will get in ;) |
This PR adds support for creating signed images bootables via MCUBoot[1].
The current available platform is the nrf52dk, but some others will be added soon.
@kaspar030 this PR already takes into account the changes to allow partitioning of the internal ROM.
[1] https://github.com/runtimeco/mcuboot/