2025 8 15 https://shelter.sirref.org/basix-syntax/ basix-syntax /basix-syntax/ Shelterfile Syntax › Basic Syntax meta-commands Take the following Dockerfile for example: hello2.txt]]> In Shelter, this can be expressed very similarly. hello2.txt]]> We have had to use two meta-commands to achieve the same functionality. The first line creates a fresh session using the alpine base image. We then import a file from the local filesystem into our image. References Context 2025 8 15 https://shelter.sirref.org/shelterfile-syntax/ shelterfile-syntax /shelterfile-syntax/ Shelterfile Syntax Shelterfiles, files with a .shl extension, are a funny mix of syntax borrowing from Dockerfiles and shell scripts. 2025 8 15 https://shelter.sirref.org/basix-syntax/ basix-syntax /basix-syntax/ Basic Syntax meta-commands Take the following Dockerfile for example: hello2.txt]]> In Shelter, this can be expressed very similarly. hello2.txt]]> We have had to use two meta-commands to achieve the same functionality. The first line creates a fresh session using the alpine base image. We then import a file from the local filesystem into our image. 2025 8 15 https://shelter.sirref.org/conditionals/ conditionals /conditionals/ Conditionals Unlike Dockerfiles, we have the possibility to introduce conditionals. Conditionals allow you to check a certain command exits with 0 and if so execute the first branch, otherwise execute the second branch. 2025 8 15 Syntax The synax is similar to many programming languages' implementation of if-then-else constructs. 2025 8 15 https://shelter.sirref.org/parallel-for/ parallel-for /parallel-for/ Parallel Loops Given Shelter's underlying mergeable structure, it can offer a rather unique model for parallel computation. Tools exist within the shell scripting world for taking scripts and arguments and running them in parallel. For example, GNU parallel or littlejohn. These tools, like Shelter, use processes for parallelism. Shelter, however, runs the processes in isolation using namespaces which gives us two advantages: It ensures that the parallel parts cannot interfere with one another. Each instantiation of the body of the loop starts from the same starting point Since each loop-body is independent, successful executions are recorded and do not need to be re-run in the event of a partial failure. After all parts of the loop have completed, the resulting data is merged and execution continues from there. By providing this as an extra feature, if users do not like these semantics, they can always opt to use a more generic tool for their parallelism too! 2025 8 15 Syntax For now, parallel for-loops are rather simplistic. #file }]]> You may introduce a variable, here file, which will take on the value in your referenced using # and will be replaced accordingly. Backlinks Related 2025 7 21 https://shelter.sirref.org/shelter-0002/ shelter-0002 /shelter-0002/ Shelter Meta-commands When developing inside Shelter, most commands are sent straight to a shell how to configure shelter To interact with the meta aspects of Shelter you can prefix your command with @. If you type just @ then you will get a list of possible commands. 2025 7 21 https://shelter.sirref.org/at-session/ at-session /at-session/ @ session Sessions are your primary way of controlling different environments. You can think of them as git branches pointing at specific instances of data and code. By default, you will start shelter in the main session. Sessions must have alpine This command will create a fresh session with no history called experiment-1 using the Debian base image. This is a similar command, but the name will be randomly generated and is ensured to be unique. It is a bit like having a Dockerfile with a FROM debian at the top. Will list all the currently available sessions. When no --image is specified, this will try to switch you into an existing session if one exists, otherwise it will create a new session based on the current session. 2025 7 21 https://shelter.sirref.org/at-undo/ at-undo /at-undo/ @ undo The @ undo command will move your session back one commit into the past. You may specify a number of commits to move, for example @ undo n to move back n commits. If there are less than n commits, we will stop going backwards. 2025 7 21 https://shelter.sirref.org/at-how/ at-how /at-how/ @ how The @ how <path> command allows you to ask Shelter to explain how a file came to be. Which executables wrote to the file, and what files did those executables read. This is an approximation to the provenance of a piece of data inside a Shelter session. 2025 7 21 https://shelter.sirref.org/at-import/ at-import /at-import/ @ import An experimental command for importing arbitrary URIs into Shelter. Shelter Programmers 2025 11 16 https://shelter.sirref.org/shelter-log-001/ shelter-log-001 /shelter-log-001/ Import statements Welcome to the first Shelter log! In this log I will explain the work I have been doing to add an @ import meta-command. The syntax thus far is relatively straightforward. ]]> There are a few key principles underlying @ import. It should be at least as expressive as Docker's COPY ... command. It should deal with a wider variety of import sources. In addition to local https:// archives over ssh etc. Imports, where possible, should be catalogued and shared across sessions. Shelter Programmers 2025 11 16 URIs as Sources branch already allows users to import local files and directories into their session. For example: By allowing arbitrary URIs in the import command, we hope to force users away from downloading data via curl or python scripts. Lifting these side-effectful imports into Shelter allows us to manage the data in a much cleaner fashion. Shelter Programmers 2025 11 16 Scripts as Sources One idea I had was to allow users to invoke scripts as there "import source". Something like: And using the same eBPF tracing we use during normal execution we can capture a fairly good idea of the tools and files needed to perform an import. Shelter Programmers 2025 11 16 Sharing Imports One example of better data management is that Shelter imports can be When a user imports some data, they can optionally name that piece of data. For example: Shelter stores that away and makes a link between the name, the URI and the underlying data. A user may then import the same piece of data into a different session by simply writing: intent via the naming scheme, it makes it possible to then version the data and have a means by which to update said data, one could imagine: Which will retry the original URI and see if the data has changed, or if it has not. There is an implicit versioning number that users can specify in the name should they choose to. Contributions