Skip to content

Conversation

@csegarragonz
Copy link
Collaborator

@csegarragonz csegarragonz commented Oct 24, 2022

In this PR we link the definition of WASM-related variables (like the maximum memroy size, or the maximum stack size) in faasmtools/build.py to the values in our headers, removing the duplication.

Faasmtools defines the necessary runtime variables and aggregates them in a dictionary, which is then appended to the environment of the python process calling CMake. Then, CMake templates a template header file (include/wasm/WasmCommon.h.in) into include/wasm/WasmCommon.h, where all the variables shared across the three runtimes are stored.

If there's a change in the environment variables in faasm/cpp, then we have to run inv dev.cmake again (CMake won't pick up changes in the values of env. variables).

For more background information see faasm/cpp#102

@csegarragonz csegarragonz changed the title Use latest CPP toolchain Unified toolchain variables Oct 27, 2022
@@ -0,0 +1,37 @@
#pragma once
Copy link
Collaborator Author

@csegarragonz csegarragonz Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is auto-generated by CMake from WasmCommon.h.in, I am not sure whether to track it or not.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usual practice for auto-generated files is to generate them in the build directory, and never modify the input source tree. It's especially useful when you set up multi-config builds which might need different versions of the same headers. I think faasm already breaks this rule with the current protobuf configuration (I did change that in my fork at some point), but it might be good to follow for future changes

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for chipping in, was about to ask you at some point.

We could eventually change the protobuf headers as well.

Copy link
Collaborator Author

@csegarragonz csegarragonz Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up, I realised FAASM_INCLUDE_DIR points to CMAKE_LIST_DIR/include, should we eventually move to CMAKE_BINARY_DIR/include and copy the includes there?

I will now do it just for one file, but for the protobuf change for example.

Also, whilst we are at it, I personally prefer to have the project name at the begining of the path, so:

#include <faasm/wasm/WasmCommon.h>

so we could copy CMAKE_LIST_DIR/include into CMAKE_BINARY_DIR/include/faasm/include what are your thoughts?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think both should be there: CMAKE_LIST_DIR/include first, and then CMAKE_BUILD_DIR/include second. That way the main source tree files take precedence, but you still get access to generated headers. It also means you can theoretically alias names and use #include_next to get an identically named header from the build tree if you want insert a hook/wrap the generated header in some extra code

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eigenraven requesting a review from you as well see if i have overlooked something

# Faasm
${FAASM_INCLUDE_DIR}
# Put the generated include directories after the source include
# directories so that the latter take precedence
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# directories so that the latter take precedence
# directories so that the former take precedence

* values using CMake
*/

#define WASM_BYTES_PER_PAGE @FAASM_WASM_BYTES_PER_PAGE@
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a point to keeping bytes-per-page dynamic? Afaik this value is frozen at 64k in the wasm spec

)

# Populate the variables defined for cross-compiling WASM
set(FAASM_WASM_BYTES_PER_PAGE $ENV{FAASM_WASM_BYTES_PER_PAGE})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any particular reason to grab these from the environment? Usually you'd use cache variables for these sort of tunables, done with set(... CACHE type "docstring"), as they are less fragile if you accidentally call cmake from the wrong context

Copy link
Collaborator Author

@csegarragonz csegarragonz Oct 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal of this PR is to share this constant definitions with the CPP toolchain, and have them hard-coded only once across all the repos. IMO, even if they won't change, I'd rather have them hardcoded only once (if that doesn't add too much overhead). They are all hardcoded in faasm/cpp/faasmtools/build.py.

So, the way I thought of sharing variables between a python script (faasm/cpp/faasmtools/build.py) and CMake files was through environment variables. It feels a bit hacky, so if you have another alternative I'm happy to adopt it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they're in a python dict, you should be able to also just pass them in to cmake during configuration time via the -D flags when they're cache variables

set(FAASM_WASM_MAX_MEMORY $ENV{FAASM_WASM_MAX_MEMORY})
set(FAASM_WASM_STACK_SIZE $ENV{FAASM_WASM_STACK_SIZE})
set(FAASM_WASM_ZYGOTE_FUNC_NAME $ENV{FAASM_WASM_ZYGOTE_FUNC_NAME})
# Shared variables with the cross-compilation toolchain
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eigenraven is this more or less what you had in mind? Also, is there any cleaner way to check if any is not set?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and the conditions can probably just be if((NOT FAASM_WASM_BYTES_PER_PAGE) OR (NOT FAASM_WASM_CTORS_FUNC_NAME) ) etc. What you wrote is wrong, because you force expansion with ${}, so it would evaluate to NOT DEFINED CACHE{65536} - when passing variables "by reference" to cmake commands you always name them, without quotes or $-expansion.

Copy link
Collaborator

@Shillaker Shillaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@csegarragonz csegarragonz merged commit 51ce569 into main Oct 31, 2022
@csegarragonz csegarragonz deleted the bump-cpp branch October 31, 2022 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants