<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="/default.xsl"?>
<fr:tree xmlns:fr="http://www.forester-notes.org" xmlns:html="http://www.w3.org/1999/xhtml" xmlns:xml="http://www.w3.org/XML/1998/namespace" root="false" base-url="/">
  <fr:frontmatter>
    <fr:authors />
    <fr:date>
      <fr:year>2025</fr:year>
      <fr:month>7</fr:month>
      <fr:day>21</fr:day>
    </fr:date>
    <fr:uri>https://shelter.sirref.org/shelter-0003/</fr:uri>
    <fr:display-uri>shelter-0003</fr:display-uri>
    <fr:route>/shelter-0003/</fr:route>
    <fr:title text="Getting Started">Getting Started</fr:title>
  </fr:frontmatter>
  <fr:mainmatter>
    <html:p>After getting <fr:link href="/shelter-0005/" title="Shelterd" uri="https://shelter.sirref.org/shelter-0005/" display-uri="shelter-0005" type="local">shelter installed</fr:link>, you can start by running the <html:code>shelter</html:code> command.</html:p>
    <html:pre><![CDATA[$ shelter]]></html:pre>
    <html:p>If this is your first time running <html:code>shelter</html:code> then you will be guided to setting up some user information. To avoid having to run commands as root, shelter comes with a <fr:link href="/shelter-0005/" title="Shelterd" uri="https://shelter.sirref.org/shelter-0005/" display-uri="shelter-0005" type="local"><html:code>shelterd</html:code></fr:link> command for running a daemon environment. Users then communicate with this daemon using <fr:link href="https://capnproto.org/" type="external">capnproto</fr:link>.</html:p>
  </fr:mainmatter>
  <fr:backmatter>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="References">References</fr:title>
      </fr:frontmatter>
      <fr:mainmatter />
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Context">Context</fr:title>
      </fr:frontmatter>
      <fr:mainmatter />
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Backlinks">Backlinks</fr:title>
      </fr:frontmatter>
      <fr:mainmatter>
        <fr:tree show-metadata="true" expanded="false" toc="false" numbered="false">
          <fr:frontmatter>
            <fr:authors>
              <fr:author>
                <fr:link href="https://shelter.sirref.org/The Shelter Developers/" type="external">The Shelter Developers</fr:link>
              </fr:author>
            </fr:authors>
            <fr:date>
              <fr:year>2025</fr:year>
              <fr:month>7</fr:month>
              <fr:day>21</fr:day>
            </fr:date>
            <fr:uri>https://shelter.sirref.org/index/</fr:uri>
            <fr:display-uri>index</fr:display-uri>
            <fr:route>/index/</fr:route>
            <fr:title text="Shelter">Shelter</fr:title>
          </fr:frontmatter>
          <fr:mainmatter>
            <html:p>Welcome to Shelter's documentation.</html:p>
            <html:p>Shelter is a <html:em>metashell</html:em>. A tool for managing shell sessions in a more reproducible way.</html:p>
            <html:p>To get started with Shelter, you can read the <fr:link href="/shelter-0003/" title="Getting Started" uri="https://shelter.sirref.org/shelter-0003/" display-uri="shelter-0003" type="local">getting started</fr:link> tutorial. For a more thorough understanding of what Shelter offers and how to use it, then please follow the <fr:link href="/shelter-0004/" title="Documentation" uri="https://shelter.sirref.org/shelter-0004/" display-uri="shelter-0004" type="local">documentation</fr:link>.</html:p>
            <fr:tree show-metadata="false" expanded="false" numbered="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>7</fr:month>
                  <fr:day>21</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/shelter-0001/</fr:uri>
                <fr:display-uri>shelter-0001</fr:display-uri>
                <fr:route>/shelter-0001/</fr:route>
                <fr:title text="About Shelter's Documentation">About Shelter's Documentation</fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p>Shelter's documentation is written in using <fr:link href="https://forester-notes.org/" type="external">Forester</fr:link> via <fr:link href="https://graft.sirref.org/" type="external">Graft</fr:link>.</html:p>
                <html:p>The main ethos is to divide the documentation into fairly small, standalone chunks. These can then be pieced together into documentation pages that are tightly linked.</html:p>
              </fr:mainmatter>
            </fr:tree>
            <html:p>For reference documentation, we have:</html:p>
            <html:ul>
              <html:li>
                <html:p>The <fr:link href="/shelterfile-syntax/" title="Shelterfile Syntax" uri="https://shelter.sirref.org/shelterfile-syntax/" display-uri="shelterfile-syntax" type="local">Shelterfile syntax</fr:link> -- the syntax for shelterfiles, a mishmash of Dockerfiles and shell scripts. These are useful for setting up branches for developement <![CDATA[or running (modest) pipelines.]]></html:p>
              </html:li>
              <html:li>
                <html:p>The available <fr:link href="/shelter-0002/" title="Shelter Meta-commands" uri="https://shelter.sirref.org/shelter-0002/" display-uri="shelter-0002" type="local">meta-commands</fr:link> -- these extend past shell scripts and allow users to access the more <html:em>interesting</html:em> parts of Shelter like <html:code>@ undo</html:code> or <html:code>@ merge &lt;branch&gt;</html:code> for example.</html:p>
              </html:li>
              <html:li>
                <html:p>The <fr:link href="/logs/" title="Shelter Logs" uri="https://shelter.sirref.org/logs/" display-uri="logs" type="local">development logs</fr:link> -- a set of developer logs about the activity on shelter.</html:p>
              </html:li>
            </html:ul>
          </fr:mainmatter>
        </fr:tree>
        <fr:tree show-metadata="true" expanded="false" toc="false" numbered="false">
          <fr:frontmatter>
            <fr:authors />
            <fr:date>
              <fr:year>2025</fr:year>
              <fr:month>7</fr:month>
              <fr:day>21</fr:day>
            </fr:date>
            <fr:uri>https://shelter.sirref.org/shelter-0002/</fr:uri>
            <fr:display-uri>shelter-0002</fr:display-uri>
            <fr:route>/shelter-0002/</fr:route>
            <fr:title text="Shelter Meta-commands">Shelter Meta-commands</fr:title>
          </fr:frontmatter>
          <fr:mainmatter>
            <html:p>When developing inside Shelter, most commands are sent straight to a shell <![CDATA[of your choice (see]]> <fr:link href="/shelter-0003/" title="Getting Started" uri="https://shelter.sirref.org/shelter-0003/" display-uri="shelter-0003" type="local">how to configure shelter</fr:link><![CDATA[).]]></html:p>
            <html:p>To interact with the meta aspects of Shelter you can prefix your command with <html:code>@</html:code>. If you type just <html:code>@</html:code> then you will get a list of possible commands.</html:p>
            <fr:tree show-metadata="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>7</fr:month>
                  <fr:day>21</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/at-session/</fr:uri>
                <fr:display-uri>at-session</fr:display-uri>
                <fr:route>/at-session/</fr:route>
                <fr:title text="@ session ">@ session </fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p>Sessions are your primary way of controlling different environments. You can think of them as git branches pointing at specific instances of data and code. By default, you will start shelter in the <html:code>main</html:code> session. Sessions <html:em>must</html:em> have <![CDATA[an associate "base image" (by default this is]]> <html:code>alpine</html:code><![CDATA[).]]></html:p>
                <html:pre><![CDATA[@ session experiment-1 --image=debian]]></html:pre>
                <html:p>This command will create a fresh session with no history called <html:code>experiment-1</html:code> using the Debian base image.</html:p>
                <html:pre><![CDATA[@ session --image=debian]]></html:pre>
                <html:p>This is a similar command, but the name will be randomly generated and is ensured to be unique. It is a bit like having a Dockerfile with a <html:code>FROM debian</html:code> at the top.</html:p>
                <html:pre><![CDATA[@ session]]></html:pre>
                <html:p>Will list all the currently available sessions.</html:p>
                <html:pre><![CDATA[@ session exp]]></html:pre>
                <html:p>When no <html:code>--image</html:code> is specified, this will try to switch you into an existing session if one exists, otherwise it will create a new session based on the current session.</html:p>
              </fr:mainmatter>
            </fr:tree>
            <fr:tree show-metadata="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>7</fr:month>
                  <fr:day>21</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/at-undo/</fr:uri>
                <fr:display-uri>at-undo</fr:display-uri>
                <fr:route>/at-undo/</fr:route>
                <fr:title text="@ undo ">@ undo </fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p>The <html:code>@ undo</html:code> command will move your session back one commit into the past. You may specify a number of commits to move, for example <html:code>@ undo n</html:code> to move back <html:code>n</html:code> commits. If there are less than <html:code>n</html:code> commits, we will stop going backwards.</html:p>
              </fr:mainmatter>
            </fr:tree>
            <fr:tree show-metadata="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>7</fr:month>
                  <fr:day>21</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/at-how/</fr:uri>
                <fr:display-uri>at-how</fr:display-uri>
                <fr:route>/at-how/</fr:route>
                <fr:title text="@ how ">@ how </fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p>The <html:code>@ how &lt;path&gt;</html:code> command allows you to ask <fr:link href="https://shelter.sirref.org/shelter/" type="external">Shelter</fr:link> to explain <html:em>how</html:em> a file came to be. Which executables wrote to the file, and what files did those executables read. This is an approximation to the provenance of a piece of data inside a <fr:link href="https://shelter.sirref.org/shelter/" type="external">Shelter</fr:link> session.</html:p>
              </fr:mainmatter>
            </fr:tree>
            <fr:tree show-metadata="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>7</fr:month>
                  <fr:day>21</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/at-import/</fr:uri>
                <fr:display-uri>at-import</fr:display-uri>
                <fr:route>/at-import/</fr:route>
                <fr:title text="@ import ">@ import </fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p>An experimental command for importing arbitrary URIs into <fr:link href="/index/" title="Shelter" uri="https://shelter.sirref.org/index/" display-uri="index" type="local">Shelter</fr:link>.</html:p>
                <fr:tree show-metadata="false">
                  <fr:frontmatter>
                    <fr:authors>
                      <fr:author>
                        <fr:link href="https://shelter.sirref.org/Shelter Programmers/" type="external">Shelter Programmers</fr:link>
                      </fr:author>
                    </fr:authors>
                    <fr:date>
                      <fr:year>2025</fr:year>
                      <fr:month>11</fr:month>
                      <fr:day>16</fr:day>
                    </fr:date>
                    <fr:uri>https://shelter.sirref.org/shelter-log-001/</fr:uri>
                    <fr:display-uri>shelter-log-001</fr:display-uri>
                    <fr:route>/shelter-log-001/</fr:route>
                    <fr:title text="Import statements">Import statements</fr:title>
                  </fr:frontmatter>
                  <fr:mainmatter>
                    <html:p>Welcome to the first <fr:link href="/index/" title="Shelter" uri="https://shelter.sirref.org/index/" display-uri="index" type="local">Shelter</fr:link> log! In this log I will explain the work I have been doing to add an <html:code>@ import</html:code> <fr:link href="/shelter-0002/" title="Shelter Meta-commands" uri="https://shelter.sirref.org/shelter-0002/" display-uri="shelter-0002" type="local">meta-command</fr:link>.</html:p>
                    <html:p>The syntax thus far is relatively straightforward.</html:p>
                    <html:pre><![CDATA[@ import --name=<optional-name> <URI> <DST>]]></html:pre>
                    <html:p>There are a few key principles underlying <html:code>@ import</html:code>.</html:p>
                    <html:ol>
                      <html:li>
                        <html:p>It should be at least as expressive as Docker's <html:code>COPY ...</html:code> command.</html:p>
                      </html:li>
                      <html:li>
                        <html:p>It should deal with a wider variety of import sources. In addition to local <![CDATA[file paths this could be data over HTTP (]]> <html:code>https://</html:code><![CDATA[), git repositories, zip]]>  archives over ssh etc.</html:p>
                      </html:li>
                      <html:li>
                        <html:p>Imports, where possible, should be catalogued and shared across <fr:link href="/at-session/" title="Shelter Meta-commands › @ session " uri="https://shelter.sirref.org/at-session/" display-uri="at-session" type="local">sessions</fr:link>.</html:p>
                      </html:li>
                    </html:ol>
                    <fr:tree show-metadata="false">
                      <fr:frontmatter>
                        <fr:authors>
                          <fr:author>
                            <fr:link href="https://shelter.sirref.org/Shelter Programmers/" type="external">Shelter Programmers</fr:link>
                          </fr:author>
                        </fr:authors>
                        <fr:date>
                          <fr:year>2025</fr:year>
                          <fr:month>11</fr:month>
                          <fr:day>16</fr:day>
                        </fr:date>
                        <fr:title text="URIs as Sources">URIs as Sources</fr:title>
                      </fr:frontmatter>
                      <fr:mainmatter>
                        <html:p><![CDATA[Point (1) can be dealt with by point (2). Indeed, that latest development]]>  branch already allows users to import local files and directories into their session. For example:</html:p>
                        <html:pre><![CDATA[@ import shelter.opam .
opam install . --deps-only --with-test]]></html:pre>
                        <html:p>By allowing arbitrary URIs in the import command, we hope to force users away from downloading data via <html:code>curl</html:code> or python scripts. Lifting these side-effectful imports into <fr:link href="https://shelter.sirref.org/shelter/" type="external">Shelter</fr:link> allows us to manage the data in a much cleaner fashion.</html:p>
                      </fr:mainmatter>
                    </fr:tree>
                    <fr:tree show-metadata="false">
                      <fr:frontmatter>
                        <fr:authors>
                          <fr:author>
                            <fr:link href="https://shelter.sirref.org/Shelter Programmers/" type="external">Shelter Programmers</fr:link>
                          </fr:author>
                        </fr:authors>
                        <fr:date>
                          <fr:year>2025</fr:year>
                          <fr:month>11</fr:month>
                          <fr:day>16</fr:day>
                        </fr:date>
                        <fr:title text="Scripts as Sources">Scripts as Sources</fr:title>
                      </fr:frontmatter>
                      <fr:mainmatter>
                        <html:p>One idea I had was to allow users to invoke scripts as there "import source". Something like:</html:p>
                        <html:pre><![CDATA[@ import -- python download_gedi_data.py]]></html:pre>
                        <html:p>And using the same <fr:link href="https://shelter.sirref.org/shelter-tracing/" type="external">eBPF tracing</fr:link> we use during normal execution we can capture a fairly good idea of the tools and files needed to perform an import.</html:p>
                      </fr:mainmatter>
                    </fr:tree>
                    <fr:tree show-metadata="false">
                      <fr:frontmatter>
                        <fr:authors>
                          <fr:author>
                            <fr:link href="https://shelter.sirref.org/Shelter Programmers/" type="external">Shelter Programmers</fr:link>
                          </fr:author>
                        </fr:authors>
                        <fr:date>
                          <fr:year>2025</fr:year>
                          <fr:month>11</fr:month>
                          <fr:day>16</fr:day>
                        </fr:date>
                        <fr:title text="Sharing Imports">Sharing Imports</fr:title>
                      </fr:frontmatter>
                      <fr:mainmatter>
                        <html:p>One example of better data management is that <fr:link href="https://shelter.sirref.org/shelter/" type="external">Shelter</fr:link> imports can be <![CDATA[explicitly shared across sessions via the naming mechanism (also by hash).]]></html:p>
                        <html:p>When a user imports some data, they can optionally name that piece of data. For example:</html:p>
                        <html:pre><![CDATA[@ import
  --name=belfast-trees \
  https://www.belfastcity.gov.uk/getmedia/262a1f01-f219-4780-835e-7a833bdd1e1c/odTrees.csv \
  /home]]></html:pre>
                        <html:p><fr:link href="https://shelter.sirref.org/shelter/" type="external">Shelter</fr:link> stores that away and makes a link between the name, the URI and the underlying data. A user may then import the same piece of data into a different session by simply writing:</html:p>
                        <html:pre><![CDATA[@ import --name=belfast-trees /var/lib]]></html:pre>
                        <html:p><![CDATA[The same data is shared (read-only) into this session. By asking for the user's]]>  intent via the naming scheme, it makes it possible to then <html:em>version</html:em> the data and have a means by which to update said data, one could imagine:</html:p>
                        <html:pre><![CDATA[@ update --name=belfast-trees]]></html:pre>
                        <html:p>Which will retry the original URI and see if the data has changed, or if it has not. There is an implicit versioning number that users can specify in the name should they choose to.</html:p>
                      </fr:mainmatter>
                    </fr:tree>
                  </fr:mainmatter>
                </fr:tree>
              </fr:mainmatter>
            </fr:tree>
          </fr:mainmatter>
        </fr:tree>
      </fr:mainmatter>
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Related">Related</fr:title>
      </fr:frontmatter>
      <fr:mainmatter>
        <fr:tree show-metadata="true" expanded="false" toc="false" numbered="false">
          <fr:frontmatter>
            <fr:authors />
            <fr:date>
              <fr:year>2025</fr:year>
              <fr:month>7</fr:month>
              <fr:day>21</fr:day>
            </fr:date>
            <fr:uri>https://shelter.sirref.org/shelter-0005/</fr:uri>
            <fr:display-uri>shelter-0005</fr:display-uri>
            <fr:route>/shelter-0005/</fr:route>
            <fr:title text="Shelterd">Shelterd</fr:title>
          </fr:frontmatter>
          <fr:mainmatter>
            <html:p>Shelter's daemon process and admin interface.</html:p>
          </fr:mainmatter>
        </fr:tree>
      </fr:mainmatter>
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Contributions">Contributions</fr:title>
      </fr:frontmatter>
      <fr:mainmatter />
    </fr:tree>
  </fr:backmatter>
</fr:tree>
