To start, we can just mimic `autoflake`, whose logic is relatively simple: https://github.com/PyCQA/autoflake/blob/60eeb31082bf5290341e84a008d5c832fb04fd13/autoflake.py#L664.