clidump

Converts natural language to terminal commands
Dump in markdown of the last few commits and their explanation

Inspiration

I was newly introduced to kubernetes and interacting with it through CLI. There were so many commands and I felt it would be easier if only there was a way to archive all the commands I used everyday so I can easily refer to them. My friend also regularly uses CLI at his job and felt that it would be amazing to have a natural language to CLI command translator to reduce contextual and cognitive load during work. Another use case for this tool is to make an archival of all the commands I use for my side projects so that when I return to it, I can easily get started on working on it as I will have all the cli commands I need (such as refreshing SSL certification) on my fingertips.

What it does

This tool only does 2 things but it does them very well.

It converts natural language input into appropriate CLI commands using openai LLM. Example, I can simply ask clidump to commit changes in branch and push it to remote and it will give me the git commands I can use to achieve that.
It has an archival functionality which will dump the previous N commands into a neat archive-version-number.md file with the explanation of what each command will do (generated using openai LLM).

How we built it

We mainly used golang to scaffold the tool and integrated shell along the way
We used github to version control
We set up github actions to run unit tests everytime a pull request was made
Used GPT to seek for ideas and code solutions when we ran into roadblocks

Challenges we ran into

We wanted to prefil the terminal with the command to execute for the natural language to cli command feature. The issue with that was goland is running as a child process in the bash terminal so it does not have access to the parent process. However, we quickly tried writing a bash script to see if this prefilling was possible and realized it actually was but just not in golang. So we tried to interoperate the shell script with our golang code and we were successful. The shell script ended up calling the golang function to achieve this functionality
Another problem we had was with the command archival feature. We were reading the command history from ~/.zsh_history and ~/.bash_history files. These files are persisted to disk. So the terminal was periodically flushing the commands to the files while retaining recent commands in memory. Since the golang program is a child process, it cannot access the run time memory of the terminal. Hence, when getting the past N commands, we were reading stale data. In order to overcome this issue, we launched a shell process from golang to dump the history in memory to a disk file. Then our golang program reads the commands from that disk file to get the latest commands. The triggering of this force flush can be expensive but its trade off is insignificant as we are still making a network call to the LLM to get the behavior of each of the commands. Also, we valued accuracy of our program over speed.

Accomplishments that we're proud of

We thought of this idea during the hackathon itself.
We completed building a working product
We used it during version control of our project while building for the hackathon
Definitely we are going to use this at work to boost our productivity
It solves our initial problem statement that we put out.

What we learned

Learned golang. Both of us were relatively new to the language.
Learned shell scripting.
We got exposed to a lot of low level OS problems such as shared memory models and isolation of child processes