<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="/default.xsl"?>
<fr:tree xmlns:fr="http://www.forester-notes.org" xmlns:html="http://www.w3.org/1999/xhtml" xmlns:xml="http://www.w3.org/XML/1998/namespace" root="false" base-url="/">
  <fr:frontmatter>
    <fr:authors />
    <fr:date>
      <fr:year>2025</fr:year>
      <fr:month>8</fr:month>
      <fr:day>15</fr:day>
    </fr:date>
    <fr:uri>https://shelter.sirref.org/parallel-for/</fr:uri>
    <fr:display-uri>parallel-for</fr:display-uri>
    <fr:route>/parallel-for/</fr:route>
    <fr:title text="Shelterfile Syntax › Parallel Loops "><fr:link href="/shelterfile-syntax/" title="Shelterfile Syntax" uri="https://shelter.sirref.org/shelterfile-syntax/" display-uri="shelterfile-syntax" type="local">Shelterfile Syntax</fr:link> › Parallel Loops </fr:title>
  </fr:frontmatter>
  <fr:mainmatter>
    <html:p>Given Shelter's underlying mergeable structure, it can offer a rather unique model for parallel computation. Tools exist within the shell scripting world for taking scripts and arguments and running them in parallel. For example, <fr:link href="https://www.gnu.org/software/parallel/" type="external">GNU parallel</fr:link> or <fr:link href="https://github.com/quantifyearth/littlejohn" type="external">littlejohn</fr:link>. These tools, like Shelter, use processes for parallelism. Shelter, however, runs the processes in isolation using namespaces which gives us two advantages:</html:p>
    <html:ol>
      <html:li>
        <html:p>It ensures that the parallel parts cannot interfere with one another. Each instantiation of the body of the loop starts from the same starting point <![CDATA[(whatever came just before).]]></html:p>
      </html:li>
      <html:li>
        <html:p>Since each loop-body is independent, successful executions are recorded and do not need to be re-run in the event of a partial failure.</html:p>
      </html:li>
    </html:ol>
    <html:p>After all parts of the loop have completed, the resulting data is <fr:link href="https://shelter.sirref.org/shelter-0009/" type="external">merged</fr:link> and execution continues from there. By providing this as an extra feature, if users do not like these semantics, they can always opt to use a more generic tool for their parallelism too!</html:p>
    <fr:tree show-metadata="false">
      <fr:frontmatter>
        <fr:authors />
        <fr:date>
          <fr:year>2025</fr:year>
          <fr:month>8</fr:month>
          <fr:day>15</fr:day>
        </fr:date>
        <fr:title text="Syntax">Syntax</fr:title>
      </fr:frontmatter>
      <fr:mainmatter>
        <html:p>For now, parallel for-loops are rather simplistic.</html:p>
        <html:pre><![CDATA[for file in [ a.txt, b.txt, c.txt ] {
  echo #file > #file
}]]></html:pre>
        <html:p>You may introduce a variable, here <html:code>file</html:code>, which will take on the value in your <![CDATA[list. In the commands of the loop-body (between the curly braces) variables are]]>  referenced using <html:code>#</html:code> and will be replaced accordingly.</html:p>
      </fr:mainmatter>
    </fr:tree>
  </fr:mainmatter>
  <fr:backmatter>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="References">References</fr:title>
      </fr:frontmatter>
      <fr:mainmatter />
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Context">Context</fr:title>
      </fr:frontmatter>
      <fr:mainmatter>
        <fr:tree show-metadata="true" expanded="false" toc="false" numbered="false">
          <fr:frontmatter>
            <fr:authors />
            <fr:date>
              <fr:year>2025</fr:year>
              <fr:month>8</fr:month>
              <fr:day>15</fr:day>
            </fr:date>
            <fr:uri>https://shelter.sirref.org/shelterfile-syntax/</fr:uri>
            <fr:display-uri>shelterfile-syntax</fr:display-uri>
            <fr:route>/shelterfile-syntax/</fr:route>
            <fr:title text="Shelterfile Syntax">Shelterfile Syntax</fr:title>
          </fr:frontmatter>
          <fr:mainmatter>
            <html:p>Shelterfiles, files with a <html:code>.shl</html:code> extension, are a funny mix of syntax borrowing from Dockerfiles and shell scripts.</html:p>
            <fr:tree show-metadata="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>8</fr:month>
                  <fr:day>15</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/basix-syntax/</fr:uri>
                <fr:display-uri>basix-syntax</fr:display-uri>
                <fr:route>/basix-syntax/</fr:route>
                <fr:title text="Basic Syntax ">Basic Syntax </fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p><![CDATA[A basic shelterfile will have a series of commands (some which might be]]> <fr:link href="/shelter-0002/" title="Shelter Meta-commands" uri="https://shelter.sirref.org/shelter-0002/" display-uri="shelter-0002" type="local">meta-commands</fr:link><![CDATA[).]]>  Take the following Dockerfile for example:</html:p>
                <html:pre><![CDATA[FROM alpine
COPY hello.txt hello.txt
RUN cat hello.txt > hello2.txt]]></html:pre>
                <html:p>In <fr:link href="https://shelter.sirref.org/shelter/" type="external">Shelter</fr:link>, this can be expressed very similarly.</html:p>
                <html:pre><![CDATA[@ session example1 --image=alpine
@ import file://./hello.txt hello.txt
cat hello.txt > hello2.txt]]></html:pre>
                <html:p>We have had to use two <fr:link href="/shelter-0002/" title="Shelter Meta-commands" uri="https://shelter.sirref.org/shelter-0002/" display-uri="shelter-0002" type="local">meta-commands</fr:link> to achieve the same functionality. The first line creates a fresh session using the <html:code>alpine</html:code> base image.</html:p>
                <html:p>We then <fr:link href="https://shelter.sirref.org/shelter-0006/" type="external">import</fr:link> a file from the local filesystem into our image.</html:p>
              </fr:mainmatter>
            </fr:tree>
            <fr:tree show-metadata="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>8</fr:month>
                  <fr:day>15</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/conditionals/</fr:uri>
                <fr:display-uri>conditionals</fr:display-uri>
                <fr:route>/conditionals/</fr:route>
                <fr:title text="Conditionals ">Conditionals </fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p>Unlike Dockerfiles, we have the possibility to introduce conditionals. Conditionals allow you to check a certain command exits with <html:code>0</html:code> and if so execute the first branch, otherwise execute the second branch.</html:p>
                <fr:tree show-metadata="false">
                  <fr:frontmatter>
                    <fr:authors />
                    <fr:date>
                      <fr:year>2025</fr:year>
                      <fr:month>8</fr:month>
                      <fr:day>15</fr:day>
                    </fr:date>
                    <fr:title text="Syntax">Syntax</fr:title>
                  </fr:frontmatter>
                  <fr:mainmatter>
                    <html:p>The synax is similar to many programming languages' implementation of <html:code>if-then-else</html:code> constructs.</html:p>
                    <html:pre><![CDATA[@ session example2 --image=alpine
apk add node
if (printf "13.0.0\n%s" $(node --version) | sort --version-sort --check=quiet) {
  echo "Node version greater than 13!"
} else {
  echo "Node version less than 13!"
}]]></html:pre>
                  </fr:mainmatter>
                </fr:tree>
              </fr:mainmatter>
            </fr:tree>
            <fr:tree show-metadata="false">
              <fr:frontmatter>
                <fr:authors />
                <fr:date>
                  <fr:year>2025</fr:year>
                  <fr:month>8</fr:month>
                  <fr:day>15</fr:day>
                </fr:date>
                <fr:uri>https://shelter.sirref.org/parallel-for/</fr:uri>
                <fr:display-uri>parallel-for</fr:display-uri>
                <fr:route>/parallel-for/</fr:route>
                <fr:title text="Parallel Loops ">Parallel Loops </fr:title>
              </fr:frontmatter>
              <fr:mainmatter>
                <html:p>Given Shelter's underlying mergeable structure, it can offer a rather unique model for parallel computation. Tools exist within the shell scripting world for taking scripts and arguments and running them in parallel. For example, <fr:link href="https://www.gnu.org/software/parallel/" type="external">GNU parallel</fr:link> or <fr:link href="https://github.com/quantifyearth/littlejohn" type="external">littlejohn</fr:link>. These tools, like Shelter, use processes for parallelism. Shelter, however, runs the processes in isolation using namespaces which gives us two advantages:</html:p>
                <html:ol>
                  <html:li>
                    <html:p>It ensures that the parallel parts cannot interfere with one another. Each instantiation of the body of the loop starts from the same starting point <![CDATA[(whatever came just before).]]></html:p>
                  </html:li>
                  <html:li>
                    <html:p>Since each loop-body is independent, successful executions are recorded and do not need to be re-run in the event of a partial failure.</html:p>
                  </html:li>
                </html:ol>
                <html:p>After all parts of the loop have completed, the resulting data is <fr:link href="https://shelter.sirref.org/shelter-0009/" type="external">merged</fr:link> and execution continues from there. By providing this as an extra feature, if users do not like these semantics, they can always opt to use a more generic tool for their parallelism too!</html:p>
                <fr:tree show-metadata="false">
                  <fr:frontmatter>
                    <fr:authors />
                    <fr:date>
                      <fr:year>2025</fr:year>
                      <fr:month>8</fr:month>
                      <fr:day>15</fr:day>
                    </fr:date>
                    <fr:title text="Syntax">Syntax</fr:title>
                  </fr:frontmatter>
                  <fr:mainmatter>
                    <html:p>For now, parallel for-loops are rather simplistic.</html:p>
                    <html:pre><![CDATA[for file in [ a.txt, b.txt, c.txt ] {
  echo #file > #file
}]]></html:pre>
                    <html:p>You may introduce a variable, here <html:code>file</html:code>, which will take on the value in your <![CDATA[list. In the commands of the loop-body (between the curly braces) variables are]]>  referenced using <html:code>#</html:code> and will be replaced accordingly.</html:p>
                  </fr:mainmatter>
                </fr:tree>
              </fr:mainmatter>
            </fr:tree>
          </fr:mainmatter>
        </fr:tree>
      </fr:mainmatter>
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Backlinks">Backlinks</fr:title>
      </fr:frontmatter>
      <fr:mainmatter />
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Related">Related</fr:title>
      </fr:frontmatter>
      <fr:mainmatter />
    </fr:tree>
    <fr:tree show-metadata="false" hidden-when-empty="true">
      <fr:frontmatter>
        <fr:authors />
        <fr:title text="Contributions">Contributions</fr:title>
      </fr:frontmatter>
      <fr:mainmatter />
    </fr:tree>
  </fr:backmatter>
</fr:tree>
