Command Runner

The Robbie command runner enables users to run arbitrary UNIX/Linux shell commands on a remote high-performance computing machine, including those with GPUs. For example, the command runner can run Python scripts (e.g., python file.py), fine-tune large language models (LLMs), or perform hyper-parameter turning with tools like Weights and Biases.

You can run simple command like:

• python file.py

• pip install -r requirements && python main.py

• python -c ‘import os; os.getcwd()’

And even more sophisticated commands like:

• tune run lora_finetune_single_device --config llama3/8B_lora_single_device

• wandb agent sweep_id

Prerequisites: To use the Robbie Command Runner, you need to:

• Place all files you need on the remote machine in a local directory

• Run robbie run –i and interactively build a configuration

• During the interactive setup, enter the commands to be run

• If you would like to see the detailed output from the remote machine, specify the –tail option

• Please note that commands run in sequence, without concern for the success for failure of the previous command. For example, if you specific three individual commands like this:

  • cd nowhere

  • python file.py

The second command ‘python file.py’ will run even if the command “cd nowhere” fails.

To sure that commands run only after the previous success, use the && convention.

Thus: `cd nowhere` && `python file.py`