1Password CLI: How NOT to Store Plaintext AWS Credentials or .env on Localhost

1Password CLI: How NOT to Store Plaintext AWS Credentials or .env on Localhost

No More ~/.aws/credetials

According to AWS security best practices, human users should access AWS services using short-term credentials provided by IAM Identity Center. Long-term credentials ("Access Key ID" and "Secret Access Key") created by IAM users should be avoided, especially since they are often stored in plaintext on disk: ~/.aws/credetials.

However, if you somehow have to use AWS access keys but want an extra layer of protection, 1Password CLI can help.

ref:
https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html
https://developer.1password.com/docs/cli/get-started

First, delete your local plaintext AWS credentials. Don't worry, you could generate new one any time on AWS Management Console.

rm -rf ~/.aws/credetials

Re-create aws-cli configuration file, but DO NOT provide any credentials.

aws configure

AWS Access Key ID [None]: JUST PRESS ENTER, DO NOT TYPE ANYTHING
AWS Secret Access Key [None]: JUST PRESS ENTER, DO NOT TYPE ANYTHING
Default region name [None]: ap-northeast-1
Default output format [None]: json

Edit ~/.aws/credentials:

[your-profile-name]
credential_process = sh -c "op item get "AWS Access Key" --account=my.1password.com --vault=Private --format=json --fields label=AccessKeyId,label=SecretAccessKey | jq 'map({key: .label, value: .value}) | from_entries + {Version: 1}'"

The magic is credential_process which sourcing AWS credentials from an external process: 1Password CLI's op item get command.

The one-liner script assumes you have an item named AWS Access Key in a vault named Private in 1Password, and the item has following fields:

  • AccessKeyId
  • SecretAccessKey

ref:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html
https://developer.1password.com/docs/cli/reference/management-commands/item#item-get

That's it.

When you run aws-cli commands or access AWS services from your code via aws-sdk, your terminal will prompt you to unlock 1Password with biometrics to source AWS credentials (once per terminal session). No more plaintext AWS access keys on localhost!

# aws-cli
aws s3 ls --profile=perp
aws logs tail --profile=perp --region=ap-northeast-1 /aws/containerinsights/perp-staging/application --follow

# aws-sdk
AWS_PROFILE=perp OTHER_ENV=123 ts-node src/index.ts

# serverless v4 supports credential_process by default
# serverless v3 requires installing a plugin: serverless-better-credentials
# https://github.com/thomasmichaelwallace/serverless-better-credentials
sls deploy --stage=staging --aws-profile=perp

# if you're using serverless-offline, you might need to add the following configs to serverless.yml
custom:
  serverless-offline:
    useInProcess: true

It's worth noting that if you prefer not to use 1Password, there is also a tool called aws-vault which can achieve a similar goal.

ref:
https://github.com/99designs/aws-vault

No More .env

If you would like to store .env file entirely in 1Password, try 1Password Environments.

ref:
https://developer.1password.com/docs/environments
https://developer.1password.com/docs/environments/local-env-file

Solidity: call() vs delegatecall()

Solidity: call() vs delegatecall()

tl;dr: delegatecall runs in the context of the caller contract.

The difference between call and delegatecall in Solidity relates to the execution context:

  • target.call(funcData):
    • the function reads/modifies target contract's storage
    • msg.sender is the caller contract
  • target.delegatecall(funcData)
    • the function reads/modifies caller contract's storage
    • msg.sender is the original sender == caller contract's msg.sender
// SPDX-License-Identifier: GPL-3.0-or-later
pragma solidity 0.8.24;

import "forge-std/Test.sol";

contract Target {
    address public owner;
    uint256 public value;

    function setOwnerAndValue(uint256 valueArg) public {
        owner = msg.sender;
        value = valueArg;
    }
}

contract Caller {
    address public owner;
    uint256 public value;

    function callSetOwnerAndValue(address target, uint256 valueArg) public {
        (bool success, ) = target.call(abi.encodeWithSignature("setOwnerAndValue(uint256)", valueArg));
        require(success, "call failed");
    }

    function delegatecallSetOwnerAndValue(address target, uint256 valueArg) public {
        (bool success, ) = target.delegatecall(abi.encodeWithSignature("setOwnerAndValue(uint256)", valueArg));
        require(success, "delegatecall failed");
    }
}

contract MyTest is Test {
    address sender = makeAddr("sender");
    Target target;
    Caller caller;

    function setUp() public {
        target = new Target();
        caller = new Caller();

        assertEq(target.owner(), address(0));
        assertEq(target.value(), 0);
        assertEq(caller.owner(), address(0));
        assertEq(caller.value(), 0);
    }

    function test_callSetOwnerAndValue() public {
        vm.prank(sender);
        caller.callSetOwnerAndValue(address(target), 100);

        // call modifies target contract's state, and target contract's msg.sender is caller contract
        assertEq(target.owner(), address(caller));
        assertEq(target.value(), 100);

        // caller contract's state didn't change
        assertEq(caller.owner(), address(0));
        assertEq(caller.value(), 0);
    }

    function test_delegatecallSetOwnerAndValue() public {
        vm.prank(sender);
        caller.delegatecallSetOwnerAndValue(address(target), 200);

        // target contract's state didn't change
        assertEq(target.owner(), address(0));
        assertEq(target.value(), 0);

        // delegatecall runs in the context of caller contract, so msg.sender is sender
        assertEq(caller.owner(), sender);
        assertEq(caller.value(), 200);
    }
}

ref:
https://medium.com/0xmantle/solidity-series-part-3-call-vs-delegatecall-8113b3c76855

Solidity: Multicall - Aggregate Multiple Contract Calls

Solidity: Multicall - Aggregate Multiple Contract Calls

There are different implementations of multicall:

In the following section, we will use Multicaller as an example to illustrate the process.

The main idea of Multicaller is to aggregate multiple contract function calls into a single one. It's usually used to batch contract reads from off-chain apps. However, it could also be used to batch contract writes.

Multiple Contract Reads

import { defaultAbiCoder } from "ethers/lib/utils"

class Liquidator {
    async fetchIsLiquidatableResults(
        marketId: number,
        positions: Position[],
    ) {
        const price = await this.pythService.fetchPythOraclePrice(marketId)

        const targets = new Array(positions.length).fill(this.exchange.address)
        const data = positions.map(position =>
            this.exchange.interface.encodeFunctionData("isLiquidatable", [
                marketId,
                position.account,
                price,
            ]),
        )
        const values = new Array(accountMarkets.length).fill(0)

        return await this.multicaller.callStatic.aggregate(targets, data, values)
    }

    async start() {
        const positions = await this.fetchPositions(marketId)
        const results = await this.fetchIsLiquidatableResults(marketId, positions)

        for (const [i, result] of results.entries()) {
            const isLiquidatable = defaultAbiCoder.decode(["bool"], result)[0]
            const position = positions[i]
            console.log(${position.account} isLiquidatable: ${isLiquidatable})
        }
    }
}

ref:
https://github.com/Vectorized/multicaller/blob/main/API.md#aggregate

Multiple Contract Writes

It requires the target contract is compatible with Multicaller if the target contract needs to read msg.sender.

// SPDX-License-Identifier: MIT
pragma solidity >=0.8.0;

import { LibMulticaller } from "multicaller/LibMulticaller.sol";

contract MulticallerSenderCompatible {
    function _sender() internal view virtual returns (address) {
        return LibMulticaller.sender();
    }
}

contract Exchange is MulticallerSenderCompatible {
    function openPosition(OpenPositionParams calldata params) external returns (int256, int256) {
        address taker = _sender();
        return _openPositionFor(taker, params);
    }
}
class Bot {
    async openPosition() {
        const targets = [
            this.oracleAdapter.address,
            this.exchange.address,
        ]
        const data = [
            this.oracleAdapter.interface.encodeFunctionData("updatePrice", [priceId, priceData]),
            this.exchange.interface.encodeFunctionData("openPosition", [params]),
        ]
        const values = [
            BigNumber.from(0),
            BigNumber.from(0),
        ]

        // update oracle price first, then open position
        const tx = await this.multicaller.connect(taker).aggregateWithSender(targets, data, values)
        await tx.wait()
    }
}

ref:
https://github.com/Vectorized/multicaller/blob/main/API.md#aggregatewithsender

Demystifying Solidity ABI Encoding

Demystifying Solidity ABI Encoding

There are some encode/decode functions in Solidity, for instance:

  • abi.encode() will concatenate all values and add padding to fit into 32 bytes for each value.
    • To integrate with other contracts, you should use abi.encode().
  • abi.encodePacked() will concatenate all values in the exact byte representations without padding.
    • If you only need to store it, you should use abi.encodePacked() since it's smaller.
  • abi.encodeWithSignature() is mainly used to call functions in another contract.
  • abi.encodeCall() is the type-safe version of abi.encodeWithSignature(), required 0.8.11+.
pragma solidity >=0.8.19;

import { IERC20 } from "openzeppelin-contracts/contracts/token/ERC20/IERC20.sol";
import "forge-std/Test.sol";

contract MyTest is Test {
    function test_abi_encode() public {
        bytes memory result = abi.encode(uint8(1), uint16(2), uint24(3));
        console.logBytes(result);
        // 0x000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000003
        // total 32 bytes * 3 = 96 bytes
    }

    function test_abi_encodePacked() public {
        bytes memory resultPacked = abi.encodePacked(uint8(1), uint16(2), uint24(3));
        console.logBytes(resultPacked);
        // 0x010002000003
        // total 1 byte + 2 bytes + 3 bytes = 6 bytes
    }

    function test_abi_encodeWithSignature() public {
        address weth = 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2;
        address vitalik = 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045;
        bytes memory data = abi.encodeWithSignature("balanceOf(address)", vitalik);
        console.logBytes(data);
        (bool success, bytes memory result) = weth.call(data);
        console.logBool(success);
        console.logUint(abi.decode(result, (uint256)));
    }

    function test_abi_encodeCall() public {
        address weth = 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2;
        address vitalik = 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045;
        bytes memory data = abi.encodeCall(IERC20.balanceOf, (vitalik));
        console.logBytes(data);
        (bool success, bytes memory result) = weth.call(data);
        console.logBool(success);
        console.logUint(abi.decode(result, (uint256)));
    }
}
forge test --mc "MyTest" -vv --fork-url https://rpc.flashbots.net

ref:
https://github.com/AmazingAng/WTF-Solidity/tree/main/27_ABIEncode
https://trustchain.medium.com/abi-functions-explained-in-solidity-bd93cf88bdf2

Build Docker Images using Google Cloud Build from Your GitHub Repository

Build Docker Images using Google Cloud Build from Your GitHub Repository

Triggering a Docker image build on Google Cloud Build when pushing commits to a GitHub repository.

Create Google Cloud Build Configuration File

Create a cloudbuild.yaml in the root folder:

substitutions:
  _REGION: us-west1
  _REPOSITORY: your-repo
  _BRANCH_TAG: ${BRANCH_NAME////-} # Replace / with - in branch names

steps:
  - id: my-app
    name: gcr.io/cloud-builders/docker
    args:
      [
        "build",
        "--cache-from",
        "${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPOSITORY}/my-app:${_BRANCH_TAG}",
        "-t",
        "${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPOSITORY}/my-app:${_BRANCH_TAG}",
        "-t",
        "${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPOSITORY}/my-app:$SHORT_SHA",
        "docker/my-blog/",
      ]
    waitFor: ["-"]

# Cloud Build only pushes tags listed here
images:
  - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPOSITORY}/my-app:$SHORT_SHA
  - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPOSITORY}/my-app:${_BRANCH_TAG}

The above config will store your Docker images in Artifact Registry. If you want to push images to another region, simply rename us-west1-docker.pkg.dev to something like asia-east1-docker.pkg.dev.

ref:
https://cloud.google.com/build/docs/configuring-builds/create-basic-configuration
https://cloud.google.com/build/docs/building/store-artifacts-in-artifact-registry

Configure Google Cloud Build with Your GitHub Repository

Go to Google Cloud Dashboard > Google Cloud Build > Triggers > Create trigger:

  • Region: Global
  • Event: Push to a branch
  • Source: 1st Gen
  • Repository:
    • Connect new repository
    • Source code management provider: GitHub (Cloud Build GitHub App)
    • Install Google Cloud Build GitHub app
    • Only select repositories
  • Branch: ^main$
  • Type: Autodetected
  • Location: Repository

That's it, now you could push some commits to your GitHub repository.

ref:
https://console.cloud.google.com/cloud-build/triggers
https://console.cloud.google.com/cloud-build/builds