What is the problem?
We have a service with a large cellular architecture. As we are migrating to CDK from LPT we need to use the Vpc.fromLookup method at least once per account. The size of the generated cdk.context.json exceeds 300K and will only keep getting larger. At this scale the process fails with E2BIG (environment string too big) when CDK code in this location attempts to pass the stringified contents of cdk.context.json to the spawned subprocess via the CDK_CONTEXT_JSON environment.
Environment strings in Linux are generally limited to 128K size and cannot be changed without recompiling the kernel!
MAX_ARG_STRLEN is a constant defined as PAGESIZE*32 in include/uapi/linux/binfmts.h. Its value cannot be changed without recompiling the kernel.
What options do we have to allow using Vpc.fromLookup which is the recommended way to do this and avoid such a roadblock?
Reproduction Steps
Add sufficient context to your cdk.context.json to exceed the limits, in our case we have a 300K cdk.context.json file. Since this is essentially a hash map of arbitrary values one could fill much of that with garbage random values reach the breaking limit.
Then try to build/synthesize your app which would invoke the CDK_CONTEXT_JSON passing mechanism for exec.
What did you expect to happen?
Build and synthesize a CDK cloud service.
What actually happened?
The call will fail with spawn E2BIG and process exits with 1.
CDK CLI Version
1.145.0 (build 9f96380)
Framework Version
No response
Node.js Version
12
OS
Amazon Linux 2
Language
Typescript
Language Version
No response
Other information
No response
What is the problem?
We have a service with a large cellular architecture. As we are migrating to CDK from LPT we need to use the Vpc.fromLookup method at least once per account. The size of the generated cdk.context.json exceeds 300K and will only keep getting larger. At this scale the process fails with E2BIG (environment string too big) when CDK code in this location attempts to pass the stringified contents of cdk.context.json to the spawned subprocess via the CDK_CONTEXT_JSON environment.
Environment strings in Linux are generally limited to 128K size and cannot be changed without recompiling the kernel!
MAX_ARG_STRLEN is a constant defined as PAGESIZE*32 in include/uapi/linux/binfmts.h. Its value cannot be changed without recompiling the kernel.
What options do we have to allow using Vpc.fromLookup which is the recommended way to do this and avoid such a roadblock?
Reproduction Steps
Add sufficient context to your cdk.context.json to exceed the limits, in our case we have a 300K cdk.context.json file. Since this is essentially a hash map of arbitrary values one could fill much of that with garbage random values reach the breaking limit.
Then try to build/synthesize your app which would invoke the CDK_CONTEXT_JSON passing mechanism for exec.
What did you expect to happen?
Build and synthesize a CDK cloud service.
What actually happened?
The call will fail with
spawn E2BIGand process exits with 1.CDK CLI Version
1.145.0 (build 9f96380)
Framework Version
No response
Node.js Version
12
OS
Amazon Linux 2
Language
Typescript
Language Version
No response
Other information
No response