<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://ubarsc.github.io//feed.xml" rel="self" type="application/atom+xml" /><link href="https://ubarsc.github.io//" rel="alternate" type="text/html" /><updated>2026-04-16T03:58:59+00:00</updated><id>https://ubarsc.github.io//feed.xml</id><title type="html">UBARSC - The UB&amp;amp;A Remote Sensing Centre</title><subtitle>Main website for ubarsc</subtitle><entry><title type="html">TuiView 1.3.7 released</title><link href="https://ubarsc.github.io//update/2026/04/16/tuiview-1.3.7.html" rel="alternate" type="text/html" title="TuiView 1.3.7 released" /><published>2026-04-16T00:00:00+00:00</published><updated>2026-04-16T00:00:00+00:00</updated><id>https://ubarsc.github.io//update/2026/04/16/tuiview-1.3.7</id><content type="html" xml:base="https://ubarsc.github.io//update/2026/04/16/tuiview-1.3.7.html"><![CDATA[<p><a href="https://tuiview.org/">TuiView</a> 1.3.7 has been released.</p>

<p>This is mainly a bugfix release. There have been some fixes to 
the way TuiView decides an image is thematic, vector labels are now
always within their polygon, and new viewers always now show the
current geolinked extent. There has been a fix for the <code class="language-plaintext highlighter-rouge">tuiview</code> process not 
exiting on Wayland, however other Qt/Wayland problems remain such
as dockable windows being unable to be moved. Until this is fixed in Qt,
using X with TuiView is recommended under Linux.</p>

<p>For more information see the <a href="https://github.com/ubarsc/tuiview/blob/master/CHANGES.txt">list of changes</a>.</p>]]></content><author><name></name></author><category term="update" /><summary type="html"><![CDATA[TuiView 1.3.7 has been released.]]></summary></entry><entry><title type="html">Accessing Arrays efficiently in Numba</title><link href="https://ubarsc.github.io//tutorial/2026/04/15/accessing-arrays-efficiently-numba.html" rel="alternate" type="text/html" title="Accessing Arrays efficiently in Numba" /><published>2026-04-15T00:00:00+00:00</published><updated>2026-04-15T00:00:00+00:00</updated><id>https://ubarsc.github.io//tutorial/2026/04/15/accessing-arrays-efficiently-numba</id><content type="html" xml:base="https://ubarsc.github.io//tutorial/2026/04/15/accessing-arrays-efficiently-numba.html"><![CDATA[<h1 id="introduction">Introduction</h1>

<p>Most modern computers come with a memory cache. This holds a copy
of memory chunks most recently used and can be <a href="https://www.hp.com/us-en/shop/tech-takes/what-is-cache-memory">up to 100 times</a>
faster than normal memory. There are in fact often multiple levels of cache
but in this discussion we just assume there is just one.</p>

<p>Each time you access memory that is not in cache (known as a cache miss),
the CPU must go to main memory to retrieve the requested item. As
it does this, it makes a prediction that the items immediately following
will be the next ones requested, and loads a block of those into the
on-chip cache at the same time (with very little extra overhead). If the
prediction turns out to be correct, this means that those subsequent
requests are met from the (much faster) on-chip cache, rather than requiring
more requests to main memory.</p>

<p>In order to gain the most benefit from this, we should generally try to ensure
that our processing matches the predictions the cache system is making,
and so we should try to process in the order in which data is being held in
main memory.</p>

<p>Directly monitoring this aspect of performance tends to be quite difficult.
Confounding the analysis is the fact that operating systems 
tend to show that the CPU is ‘busy’ while waiting for
data to come from main memory. The only way to determine whether
a particular implementation is fast or not is to run it and
time how long it takes. Using how ‘busy’ the CPU is might not
give you an accurate picture.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A note about multi threading: Run multiple threads at once 
generally makes it harder for the cache to run efficiently 
since the memory requests become less predicable. Software also 
tends to have to add more locking and checking when multiple
threads are enabled and this overhead can affect the speed
of your code.
</code></pre></div></div>

<p>The below assumes there is no other competing processing
running on your hardware, results will vary depending on
what other things are going on in your system.</p>

<h1 id="this-simple-1d-case">This simple 1D case</h1>

<p>Assume you are looping over a 1 dimensional array from the start
to the end. The array is 64KB (of 8 byte floats) and we 
assume the cache is 16KB:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[1,2,3.....2047,2048,2049,......4095,4096,4097.......8191]
 ^     ^        ^                    ^      ^
Cache  Cache    Cache                Cache  Cache
miss   hit      miss                 miss   hit
</code></pre></div></div>

<p>We are assuming that the CPU is implementing a simple caching 
system without any heuristics. When your code accesses the first
element there will likely be a cache miss. However for the next 
2048 elements the data will likely be in the cache already
and calculations will progress faster. But for element 2048 we
run out of data that is in the cache so that element will be 
slower to acccess while the value is fetched from main memory 
and the cache updated. Element 2049 will be relatively quick 
again until we hit element 4096.
So there will be 4 cache misses as you process the array in
sequential order. What happens when you process the array out of
order? If you processed element 0, first then element 2048,
then back to element 1? Well that could end up having a cache 
miss for every access - 2048 times slower!</p>

<p>So it is best to access elements that are next to each other.</p>

<h1 id="the-2-dimensional-case">The 2 dimensional case</h1>

<p>Things get a bit more complex with the multidimensional case. By 
default <a href="https://numpy.org/devdocs/dev/internals.html">numpy arrays are laid out in the “C” order</a>.
C uses row-major order, for an array with shape=(4, 5) this means:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>array[0, 0], array[0, 1], array[0, 2], array[0, 3], array[1, 0], array[1, 1], array[1, 2].....
</code></pre></div></div>

<p>As you can see the last axis is grouped together in the array. So 
it makes the most sense to loop through each row so you are always
accessing the next element:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">data</span><span class="p">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
    <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">data</span><span class="p">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]):</span>
        <span class="nb">sum</span> <span class="o">+=</span> <span class="n">data</span><span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="n">j</span><span class="p">]</span>
</code></pre></div></div>

<p>If you were to reverse the order of these loops you would be ‘jumping’
about in memory by <code class="language-plaintext highlighter-rouge">data.shape[1]</code> - not a huge deal with small arrays
but with larger ones this slowdown could be significant.</p>

<h1 id="the-multidimensional-case-and-transposing">The multidimensional case and transposing</h1>

<p>This only gets more important if you have arrays with many dimensions.
Always loop through your array so the tightest loop is through the 
last axis, the second tightest loop through the second to last axis etc 
(assuming C layout).</p>

<p>This is how <a href="https://numpy.org/devdocs/reference/c-api/iterator.html">numpy does it internally</a>.</p>

<p>But what happens when your algorithm requires visiting the axes in 
a different order? This is where <code class="language-plaintext highlighter-rouge">numpy.transpose</code> comes in. It is often 
worthwhile transposing the order of your axes first so that they are in
the most optimal order for your algorithm.
But won’t this create extra overhead? It does, but it is not as bad as
you might think. Firstly the input array is read by numpy in a sequential
order. Secondly, writing back to memory appears to happen “in the background”
so usually by the time you are reading the values they have been written 
to memory and the CPU does not have to wait. Often there are fewer cache misses
in total doing this.
Will this help your particular use case? You’d have to benchmark the transpose
and not transposed case and see what is faster. For big arrays it usually 
is faster.</p>

<h1 id="conclusion">Conclusion</h1>

<p>Not all locations in memory take equal time to access. Because of the cache,
most times it is the very next element from the previous one that will be fastest.
You will need to think about this especially for multidimensional arrays.
Note that processing using threads reduces predictability and often takes
more CPU cycles for the same result. This is why Numba’s <code class="language-plaintext highlighter-rouge">prange</code> usually does
not give expected speedups. 
Transposing your input array so it is in the correct order is often leads 
to a significant speedup, but the only way to be sure is to run timings
for the various approaches.</p>]]></content><author><name></name></author><category term="tutorial" /><summary type="html"><![CDATA[Introduction]]></summary></entry><entry><title type="html">Using numpy masked arrays</title><link href="https://ubarsc.github.io//tutorial/2026/04/01/using-masked-arrays.html" rel="alternate" type="text/html" title="Using numpy masked arrays" /><published>2026-04-01T00:00:00+00:00</published><updated>2026-04-01T00:00:00+00:00</updated><id>https://ubarsc.github.io//tutorial/2026/04/01/using-masked-arrays</id><content type="html" xml:base="https://ubarsc.github.io//tutorial/2026/04/01/using-masked-arrays.html"><![CDATA[<h1 id="introduction">Introduction</h1>

<p>Dealing with numpy arrays that have missing data is a challenge. When 
calculating statistics on them or otherwise processing data in them 
you need to skip elements that are missing - often they are some 
very high (or low) nodata value and will skew the result.</p>

<h1 id="using-nans">Using NaNs</h1>

<p>In floating point data, there is a special number: NaN (“not a number”)
that you can use to signify that this value can’t be processed for whatever reason.
Numpy has a collection of functions (they all start with “nan”) that ignore
any NaNs in your data. You an also test for individual elements being NaN
with <code class="language-plaintext highlighter-rouge">numpy.isnan</code>.
However, what happens if you are dealing with integer data? Well, setting
integer elements to NaN fails. The alternative is to convert the whole
integer array you are dealing with to float and back again. This adds
time and memory use. Also performing operations on floats is slower
than on integers.</p>

<h1 id="numpy-masked-arrays">Numpy Masked Arrays</h1>

<p>There is an alternative - <a href="https://numpy.org/doc/stable/reference/maskedarray.html">numpy masked arrays</a>. These
are arrays that also store a mask of where data is not valid:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">numpy</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">x</span> <span class="o">=</span> <span class="n">numpy</span><span class="p">.</span><span class="n">array</span><span class="p">([[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">9999</span><span class="p">],</span> <span class="p">[</span><span class="o">-</span><span class="mi">980</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">9</span><span class="p">],</span> <span class="p">[</span><span class="mi">11</span><span class="p">,</span> <span class="mi">61</span><span class="p">,</span> <span class="mi">9923</span><span class="p">]])</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">mx</span> <span class="o">=</span> <span class="n">numpy</span><span class="p">.</span><span class="n">ma</span><span class="p">.</span><span class="n">masked_array</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">mask</span><span class="o">=</span><span class="p">[[</span><span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span> <span class="bp">True</span><span class="p">],</span> <span class="p">[</span><span class="bp">True</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">],</span> <span class="p">[</span><span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span> <span class="bp">True</span><span class="p">]])</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">mx</span>
<span class="n">masked_array</span><span class="p">(</span>
  <span class="n">data</span><span class="o">=</span><span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="o">--</span><span class="p">],</span>
        <span class="p">[</span><span class="o">--</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">9</span><span class="p">],</span>
        <span class="p">[</span><span class="mi">11</span><span class="p">,</span> <span class="mi">61</span><span class="p">,</span> <span class="o">--</span><span class="p">]],</span>
  <span class="n">mask</span><span class="o">=</span><span class="p">[[</span><span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span>  <span class="bp">True</span><span class="p">],</span>
        <span class="p">[</span> <span class="bp">True</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">],</span>
        <span class="p">[</span><span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span>  <span class="bp">True</span><span class="p">]],</span>
  <span class="n">fill_value</span><span class="o">=</span><span class="mi">999999</span><span class="p">)</span>
</code></pre></div></div>

<p>It is important to note the <code class="language-plaintext highlighter-rouge">mask</code> parameter is <code class="language-plaintext highlighter-rouge">True</code> where the data <em>is masked</em>.
Masked arrays support <a href="https://numpy.org/doc/stable/reference/maskedarray.baseclass.html#maskedarray-methods">many of the numpy methods</a>:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="n">mx</span><span class="p">.</span><span class="nb">max</span><span class="p">()</span>
<span class="n">np</span><span class="p">.</span><span class="n">int64</span><span class="p">(</span><span class="mi">61</span><span class="p">)</span>
</code></pre></div></div>

<p>Note how the masked out values aren’t used in the calculations. You can also turn 
a masked array back into a normal array using the <code class="language-plaintext highlighter-rouge">filled</code> method:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="n">mx</span><span class="p">.</span><span class="n">filled</span><span class="p">(</span><span class="o">-</span><span class="mi">99</span><span class="p">)</span>
<span class="n">array</span><span class="p">([[</span>  <span class="mi">1</span><span class="p">,</span>   <span class="mi">5</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">],</span>
       <span class="p">[</span><span class="o">-</span><span class="mi">99</span><span class="p">,</span>   <span class="mi">7</span><span class="p">,</span>   <span class="mi">9</span><span class="p">],</span>
       <span class="p">[</span> <span class="mi">11</span><span class="p">,</span>  <span class="mi">61</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">]])</span>
</code></pre></div></div>

<p>If your data has a single value that represents “no data” then you can pass this in
to the <code class="language-plaintext highlighter-rouge">masked_values</code> function to create a masked array where that value is masked:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">x</span> <span class="o">=</span> <span class="n">numpy</span><span class="p">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">,</span> <span class="mi">23</span><span class="p">,</span> <span class="mi">78</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">])</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">mx</span> <span class="o">=</span> <span class="n">numpy</span><span class="p">.</span><span class="n">ma</span><span class="p">.</span><span class="n">masked_values</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">)</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">mx</span>
<span class="n">masked_array</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="o">--</span><span class="p">,</span> <span class="mi">23</span><span class="p">,</span> <span class="mi">78</span><span class="p">,</span> <span class="o">--</span><span class="p">],</span>
             <span class="n">mask</span><span class="o">=</span><span class="p">[</span><span class="bp">False</span><span class="p">,</span>  <span class="bp">True</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span> <span class="bp">False</span><span class="p">,</span>  <span class="bp">True</span><span class="p">],</span>
       <span class="n">fill_value</span><span class="o">=-</span><span class="mi">99</span><span class="p">)</span>
</code></pre></div></div>

<h1 id="notes-on-using-numba-with-masked-arrays">Notes on using Numba with masked arrays</h1>

<p>Numba doesn’t know anything about masked arrays - you get an <code class="language-plaintext highlighter-rouge">Unsupported array type: numpy.ma.MaskedArray</code> error when you pass one in to a Numba function. However, 
a masked array is made up of two normal arrays: the data and the mask. You can 
pass these in separately to a Numba function:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">@</span><span class="n">njit</span>
<span class="k">def</span> <span class="nf">docalc</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">mask</span><span class="p">):</span>
    <span class="n">tot</span> <span class="o">=</span> <span class="mi">0</span>
    <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">data</span><span class="p">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
        <span class="k">if</span> <span class="ow">not</span> <span class="n">mask</span><span class="p">[</span><span class="n">x</span><span class="p">]:</span>
            <span class="n">tot</span> <span class="o">+=</span> <span class="n">data</span><span class="p">[</span><span class="n">x</span><span class="p">]</span>
    <span class="k">return</span> <span class="n">tot</span>
    
<span class="n">x</span> <span class="o">=</span> <span class="n">numpy</span><span class="p">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">,</span> <span class="mi">23</span><span class="p">,</span> <span class="mi">78</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">])</span>
<span class="n">mx</span> <span class="o">=</span> <span class="n">numpy</span><span class="p">.</span><span class="n">ma</span><span class="p">.</span><span class="n">masked_values</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="o">-</span><span class="mi">99</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">docalc</span><span class="p">(</span><span class="n">mx</span><span class="p">.</span><span class="n">data</span><span class="p">,</span> <span class="n">mx</span><span class="p">.</span><span class="n">mask</span><span class="p">)</span>
</code></pre></div></div>

<p>However, if your data just has a single “no data” value it may
be easier just to pass this value in and compare each element to it
instead of using masked arrays.</p>

<h1 id="conclusion">Conclusion</h1>

<p>Numba masked arrays can be a useful tool when dealing with missing
data. They provide a lighter weight alternative to conversion to float
arrays and setting NaN.</p>]]></content><author><name></name></author><category term="tutorial" /><summary type="html"><![CDATA[Introduction]]></summary></entry><entry><title type="html">Distributing our software - conda-forge, not pip/PyPI</title><link href="https://ubarsc.github.io//tutorial/2026/03/14/conda-not-pip.html" rel="alternate" type="text/html" title="Distributing our software - conda-forge, not pip/PyPI" /><published>2026-03-14T00:00:00+00:00</published><updated>2026-03-14T00:00:00+00:00</updated><id>https://ubarsc.github.io//tutorial/2026/03/14/conda-not-pip</id><content type="html" xml:base="https://ubarsc.github.io//tutorial/2026/03/14/conda-not-pip.html"><![CDATA[<h1 id="introduction">Introduction</h1>

<p>Several of the packages available from the UBARSC group are most easily installed using the <code class="language-plaintext highlighter-rouge">conda</code> command. Some users are more familiar with using the <code class="language-plaintext highlighter-rouge">pip</code> command to download &amp; install Python packages, and have wondered why we do not make our packages available in this way. What follows is a discussion of the major issues and reasons for this choice. Note that this is not a tutorial on the usage of either system.</p>

<h1 id="what-are-pip-conda-pypi-and-conda-forge">What Are Pip, Conda, PyPI and Conda-forge</h1>

<p>The <code class="language-plaintext highlighter-rouge">pip</code> command is a tool for installing Python packages. It understands the conventions used for distributing Python packages, and can be used to perform the installation directly from a source repository or <code class="language-plaintext highlighter-rouge">.tar.gz</code> distribution file. By default it will install things from the Python Package Index website (PyPI), handling all downloading directly. It understands when other Python packages are needed as dependencies for the target package, and will download and install those as well.</p>

<p>The <code class="language-plaintext highlighter-rouge">conda</code> command is a package manager with a much broader scope. It can install anything which has been packaged up for conda-based installation, and has no particular bias towards Python packages or anything else.</p>

<p>The PyPI website is a large repository for distributing Python packages, aimed at making it easy to download and install anywhere. It distributes both pure Python source, and pre-built “wheel” files (<code class="language-plaintext highlighter-rouge">.whl</code>) which might include compiled C-extension modules. It is the default source for packages to be installed by the <code class="language-plaintext highlighter-rouge">pip</code> command.</p>

<p>The conda ecosystem was developed by a private company, Anaconda (formerly Continuum Analytics), which sells (among other things) their services for software packaging and distribution. They have worked collaboratively with the open source community to maintain the conda tool and specifications, and host alternative community distribution channels. The major such alternative channel is <code class="language-plaintext highlighter-rouge">conda-forge</code>, and it is through this channel that we at UBARSC distribute our software. The <code class="language-plaintext highlighter-rouge">conda-forge</code> channel includes distributions for a wide range of data science related software, including binary distributions for many tools and libraries unrelated to Python.</p>

<h1 id="pippypi-vs-condaconda-forge">Pip/PyPI vs conda/conda-forge</h1>

<p>The PyPI repository is fine for distributing packages which are pure Python. All that is required is the Python source code, and some small text configuration files. However, distributing packages
which are written in something other than Python is not really supported (although it can be kludged around, in some cases). A good example would be the KEA file format. The KEA library is written in
C++, and while there is also a Python binding, this is largely separate from the library code
itself. PyPI and the <code class="language-plaintext highlighter-rouge">pip</code> command don’t really have any idea what to do with all that.</p>

<p>Another more important example is the GDAL library. This is also written in C++, with an optional
Python binding, and again, PyPI/pip has little to offer.</p>

<p>For example, RIOS depends very heavily on GDAL. So, if we were to distribute RIOS though PyPI, we ought to include this dependency in its <code class="language-plaintext highlighter-rouge">requirements.txt</code> file. However, if we did that, then <code class="language-plaintext highlighter-rouge">pip</code> would try to install GDAL from PyPI. The GDAL Python bindings are present in PyPI, but this is just the bindings, not the library itself, and so GDAL would still not be installed. If we do not specify the dependency, then the situation would be just as bad, and with little guidance for the novice user.</p>

<p>So, one would have to rely on something like <code class="language-plaintext highlighter-rouge">conda</code> to install GDAL itself.</p>

<p>The GDAL bindings also require numpy. However, numpy is also fully available from PyPI. This means that, depending on what else is installed, it is possible to have a numpy from PyPI and a GDAL from conda, and they may well be compiled in ways which are incompatible at the binary level, resulting in serious conflicts at run time.</p>

<p>Things can become even more complicated when one or more packages installed from PyPI include their own copies of binaries from other libraries. A good example of this is <code class="language-plaintext highlighter-rouge">rasterio</code>, which bundles its own copy of the GDAL binaries. These may be compiled with different options than any already installed version of GDAL, potentially causing all kinds of headaches.</p>

<p>To avoid much of this complexity, we have concluded that it is simpler just to rely on <code class="language-plaintext highlighter-rouge">conda</code> to distribute our packages, and everything on which they depend.</p>

<p>There is a great deal of discussion on some of these points spread all over the Internet. <a href="https://www.anaconda.com/blog/understanding-conda-and-pip">This article</a> from Anaconda provides a brief overview (although slightly out of date).</p>

<p>As discussed in that article, <code class="language-plaintext highlighter-rouge">conda</code> and <code class="language-plaintext highlighter-rouge">pip</code> can work moderately well together, and this is useful for adding in small pure Python packages which are not available in conda. However, one does need to watch for pip’s tendency to try to install its own dependencies, and upgrade things which conda has already installed, resulting in incompatible combinations of binaries. For this reason it is strongly recommended that any package (including dependencies) which contains compiled binary files should come from <code class="language-plaintext highlighter-rouge">conda-forge</code>, and only pure Python should be installed from PyPI. Please exercise strong caution when combining them, including</p>

<ol>
  <li>being aware of the dependencies of the package you are about to install</li>
  <li>watching the install as it runs, to see whether it tells you what else it is installing</li>
</ol>

<p>This applies even when the target package being installed is from local source rather than PyPI. The ideal output of a <code class="language-plaintext highlighter-rouge">pip</code> installation should say that it installed the package itself, but that all dependencies were already satisfied. If <code class="language-plaintext highlighter-rouge">pip</code> starts over-writing things, you will probably need to discard that conda environment and start again.</p>

<h1 id="conclusion">Conclusion</h1>
<p>In short, we recommend that you build working environments using the packages on <code class="language-plaintext highlighter-rouge">conda-forge</code>, including our own packages such as RIOS, PyShepSeg, TuiView, etc., and only use pip/PyPI to install small, pure Python packages, and only if they are not available via <code class="language-plaintext highlighter-rouge">conda-forge</code>. Of course, more sophisticated users have other options, such as building things directly from source, but we assume that such users are sophisticated enough to know what they are doing.</p>]]></content><author><name></name></author><category term="tutorial" /><summary type="html"><![CDATA[Introduction]]></summary></entry><entry><title type="html">Introducing the RIOS Reaper for terminating unused AWS resources</title><link href="https://ubarsc.github.io//tutorial/2026/03/12/rios-reaper.html" rel="alternate" type="text/html" title="Introducing the RIOS Reaper for terminating unused AWS resources" /><published>2026-03-12T00:00:00+00:00</published><updated>2026-03-12T00:00:00+00:00</updated><id>https://ubarsc.github.io//tutorial/2026/03/12/rios-reaper</id><content type="html" xml:base="https://ubarsc.github.io//tutorial/2026/03/12/rios-reaper.html"><![CDATA[<h1 id="introduction">Introduction</h1>

<p><a href="https://www.rioshome.org">RIOS</a> and <a href="https://www.pyshepseg.org">PyShepSeg</a>
have support for Concurrency using AWS. RIOS <a href="https://www.rioshome.org/en/latest/concurrency.html">supports ECS and Fargate clusters</a> and PyShepSeg currently
only <a href="https://www.pyshepseg.org/en/latest/#concurrency">supports Fargate clusters</a>.
These packages attempt to clean up any resources they create. However 
there may be situations, such as termination of the main script or software
error, where these resources are not terminated.
<a href="https://github.com/gillins/rios_reaper">RIOS Reaper</a> is a tool that monitors resources created by RIOS and PyShepSeg
and notifies a user via email about anything that looks like it should
be terminated. It does this by looking for resource tags that RIOS and PyShepSeg
add as they are creating resources. It then checks EC2 instances for idle CPU 
and ECS and Fargate clusters that are stopped.
Currently, RIOS Reaper does not terminate any EC2 instances
or Fargate clusters. We feel it is safer for a user to delete these in case
they are still in use.</p>

<h1 id="installation">Installation</h1>

<p><a href="https://github.com/gillins/rios_reaper">RIOS Reaper</a> is a Lambda that runs once a day. 
It is based on <a href="https://aws.amazon.com/serverless/sam/">AWS SAM</a> and this needs to be installed
first. Any machine that is connected to the internet and logged into AWS should be
suitable for running the deployment, however we have only tested on Linux.
Further instructions are in the <a href="https://github.com/gillins/rios_reaper/blob/main/README.md">README</a>. Note 
will need to be logged onto your AWS account on the command line with sufficient parameters to install
a Lambda.</p>

<p>Since this job runs on a Lambda at a nominated time there is no extra machine to
be provisioned and you’ll only be paying for when the Lambda is running. AWS takes
care of running Lambda code, you just need to provide the function.</p>

<p>Information about your account and VPC is required. This is passed in via environment variables:</p>
<ul>
  <li>AWS_PROFILE - the name of the profile you are running under, or <code class="language-plaintext highlighter-rouge">default</code></li>
  <li>VPC_ID - the id of the VPC you want the Lambda to run under</li>
  <li>SUBNET_IDS - a comma separated list of subnet ids the Lambda is to run within</li>
  <li>EMAIL - the email address you want notifications to be sent to</li>
</ul>

<p>You may also want to make changes to <code class="language-plaintext highlighter-rouge">template.yaml</code> which is the CloudFormation
template used by AWS SAM. Check that the <code class="language-plaintext highlighter-rouge">Architecture</code> parameter is correct
for your configuration (also in <code class="language-plaintext highlighter-rouge">samconfig.toml</code>) and that the <code class="language-plaintext highlighter-rouge">Schedule</code> setting is correct for when you
want the check to be run. The region can be set in <code class="language-plaintext highlighter-rouge">samconfig.toml</code>.</p>

<p>Once this information is set, you can test running the Lambda locally:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./test-deploy.py
</code></pre></div></div>

<p>Once you are happy, deploy the Lambda like this:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./test-deploy <span class="nt">-m</span> deployed
</code></pre></div></div>

<p>To remove the Lambda:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sam delete
</code></pre></div></div>

<h1 id="conclusion">Conclusion</h1>

<p>RIOS Reaper is a relatively easy way to check that there are no “zombie” jobs
running that are costing you money. You will get an email once a day containing
details of instances and clusters that may need to be checked and terminated.</p>]]></content><author><name></name></author><category term="tutorial" /><summary type="html"><![CDATA[Introduction]]></summary></entry><entry><title type="html">Driving TuiView from Python</title><link href="https://ubarsc.github.io//tutorial/2026/03/08/tuiview-from-python.html" rel="alternate" type="text/html" title="Driving TuiView from Python" /><published>2026-03-08T00:00:00+00:00</published><updated>2026-03-08T00:00:00+00:00</updated><id>https://ubarsc.github.io//tutorial/2026/03/08/tuiview-from-python</id><content type="html" xml:base="https://ubarsc.github.io//tutorial/2026/03/08/tuiview-from-python.html"><![CDATA[<h1 id="introduction">Introduction</h1>

<p>Although <a href="https://tuiview.org/">TuiView</a> is a useful program on its own, sometimes
it may be useful to embed some of its functionality within another script. In this
way you can build a customised image viewer. 
Since TuiView is just a Python module you can access it from any Python script. However,
a certain amount of Qt/PySide knowledge is required for building user interfaces. This
tutorial is based on the <a href="https://github.com/ubarsc/tuiview/wiki/Embedding">TuiView wiki</a>.
Documentation for the TuiView internals can be found at the 
<a href="https://tuiview.readthedocs.io/">TuiView Developer Documentation</a>.</p>

<h1 id="creating-a-tuiview-widget-within-your-own-window">Creating a TuiView widget within your own window</h1>

<p>This is the simplest scenario. You have your own window, but you want one of the 
widgets to be a TuiView map:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_widget.png" alt="TuiView widget in another application" /></p>

<p>Below is the source for this application:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">from</span> <span class="nn">PySide6.QtWidgets</span> <span class="kn">import</span> <span class="n">QApplication</span><span class="p">,</span> <span class="n">QWidget</span><span class="p">,</span> <span class="n">QVBoxLayout</span>
<span class="kn">from</span> <span class="nn">tuiview</span> <span class="kn">import</span> <span class="n">viewerwidget</span><span class="p">,</span> <span class="n">viewerstretch</span>
<span class="kn">from</span> <span class="nn">osgeo</span> <span class="kn">import</span> <span class="n">gdal</span>

<span class="n">gdal</span><span class="p">.</span><span class="n">UseExceptions</span><span class="p">()</span>

<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">(</span><span class="n">sys</span><span class="p">.</span><span class="n">argv</span><span class="p">)</span>

<span class="n">ds</span> <span class="o">=</span> <span class="n">gdal</span><span class="p">.</span><span class="n">Open</span><span class="p">(</span><span class="s">'a.tif'</span><span class="p">)</span>
<span class="n">stretch</span> <span class="o">=</span> <span class="n">viewerstretch</span><span class="p">.</span><span class="n">ViewerStretch</span><span class="p">()</span>
<span class="n">stretch</span><span class="p">.</span><span class="n">setRGB</span><span class="p">()</span>
<span class="n">stretch</span><span class="p">.</span><span class="n">setBands</span><span class="p">([</span><span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">2</span><span class="p">])</span>
<span class="n">stretch</span><span class="p">.</span><span class="n">setStdDevStretch</span><span class="p">()</span>

<span class="n">w</span> <span class="o">=</span> <span class="n">QWidget</span><span class="p">()</span>

<span class="n">tmap</span> <span class="o">=</span> <span class="n">viewerwidget</span><span class="p">.</span><span class="n">ViewerWidget</span><span class="p">(</span><span class="n">w</span><span class="p">)</span>
<span class="n">w</span><span class="p">.</span><span class="n">show</span><span class="p">()</span>

<span class="c1"># Note that TuiView expects the widget to be shown before
# adding a layer
</span><span class="n">tmap</span><span class="p">.</span><span class="n">addRasterLayer</span><span class="p">(</span><span class="n">ds</span><span class="p">,</span> <span class="n">stretch</span><span class="p">)</span>

<span class="n">layout</span> <span class="o">=</span> <span class="n">QVBoxLayout</span><span class="p">()</span>
<span class="n">layout</span><span class="p">.</span><span class="n">addWidget</span><span class="p">(</span><span class="n">tmap</span><span class="p">)</span>
<span class="n">w</span><span class="p">.</span><span class="n">setLayout</span><span class="p">(</span><span class="n">layout</span><span class="p">)</span>
<span class="n">w</span><span class="p">.</span><span class="n">resize</span><span class="p">(</span><span class="mi">250</span><span class="p">,</span> <span class="mi">150</span><span class="p">)</span>
<span class="n">w</span><span class="p">.</span><span class="n">move</span><span class="p">(</span><span class="mi">300</span><span class="p">,</span> <span class="mi">300</span><span class="p">)</span>
<span class="n">w</span><span class="p">.</span><span class="n">setWindowTitle</span><span class="p">(</span><span class="s">'Simple'</span><span class="p">)</span>

<span class="n">app</span><span class="p">.</span><span class="k">exec</span><span class="p">()</span>

</code></pre></div></div>

<h1 id="creating-new-viewers">Creating new viewers</h1>

<p>If you are happy with the way the TuiView windows look, but you just want to drive
them programmatically, the recommended way to do this is to use the <code class="language-plaintext highlighter-rouge">GeolinkedViewers</code>
class:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_geolinkpython.png" alt="TuiView Geolinked Viewers" /></p>

<p>Below is the source for this application:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">sys</span>

<span class="kn">from</span> <span class="nn">PySide6.QtWidgets</span> <span class="kn">import</span> <span class="n">QApplication</span>

<span class="kn">from</span> <span class="nn">tuiview</span> <span class="kn">import</span> <span class="n">geolinkedviewers</span>
<span class="kn">from</span> <span class="nn">tuiview.viewerwidget</span> <span class="kn">import</span> <span class="n">GeolinkInfo</span>

<span class="n">app</span> <span class="o">=</span> <span class="n">QApplication</span><span class="p">(</span><span class="n">sys</span><span class="p">.</span><span class="n">argv</span><span class="p">)</span>

<span class="n">glviewers</span> <span class="o">=</span> <span class="n">geolinkedviewers</span><span class="p">.</span><span class="n">GeolinkedViewers</span><span class="p">()</span>
<span class="n">viewer1</span> <span class="o">=</span> <span class="n">glviewers</span><span class="p">.</span><span class="n">newViewer</span><span class="p">(</span><span class="s">'a.tif'</span><span class="p">)</span>
<span class="n">viewer2</span> <span class="o">=</span> <span class="n">glviewers</span><span class="p">.</span><span class="n">newViewer</span><span class="p">(</span><span class="s">'b.kea'</span><span class="p">)</span>

<span class="c1"># The first parameter is the 'id' of the sender; 
# set to 0 if not sent from inside TuiView.
# Then pass Easting, Northing and meters per pixel as zoom factor
</span><span class="n">obj</span> <span class="o">=</span> <span class="n">GeolinkInfo</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1976486</span><span class="p">,</span> <span class="o">-</span><span class="mi">3144006</span><span class="p">,</span> <span class="mi">100</span><span class="p">)</span>
<span class="n">glviewers</span><span class="p">.</span><span class="n">onMove</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>

<span class="n">app</span><span class="p">.</span><span class="k">exec</span><span class="p">()</span>
</code></pre></div></div>

<h1 id="conclusion">Conclusion</h1>

<p>Most aspects of TuiView can be automated from a Python script. This means
you can create your own custom applications that re-use TuiView functionality
without having to reinvent the wheel.</p>]]></content><author><name></name></author><category term="tutorial" /><summary type="html"><![CDATA[Introduction]]></summary></entry><entry><title type="html">New Python script to convert Python docstrings to Markdown files</title><link href="https://ubarsc.github.io//new/2026/03/03/plainpydoc2md-announce.html" rel="alternate" type="text/html" title="New Python script to convert Python docstrings to Markdown files" /><published>2026-03-03T01:55:00+00:00</published><updated>2026-03-03T01:55:00+00:00</updated><id>https://ubarsc.github.io//new/2026/03/03/plainpydoc2md-announce</id><content type="html" xml:base="https://ubarsc.github.io//new/2026/03/03/plainpydoc2md-announce.html"><![CDATA[<p>A new Python script <a href="https://github.com/ubarsc/plainpydoc2md">plainpydoc2md</a>
has been released.</p>

<p>This script reads one or more Python files and writes corresponding Markdown
files containing their docstrings, formatted in a neat and readable form.</p>

<p>It is intended as a simple alternative to using tools like Sphinx, ReadTheDocs,
or GitHub Pages, when the complexity and overhead of those tools is not justified.</p>

<p>The advantage of writing Markdown is that it can be viewed directly from a
Github repository. Thus, Github can easily serve the documentation pages
for a project, without setting up any extra systems.</p>]]></content><author><name></name></author><category term="new" /><summary type="html"><![CDATA[A new Python script plainpydoc2md has been released.]]></summary></entry><entry><title type="html">Announcing RatZarr, RIOS 2.0.9 and PyShepSeg 2.0.5</title><link href="https://ubarsc.github.io//update/2026/01/30/ratzarr-1.0.1-rios-2.0.9-pyshepseg-2.0.5.html" rel="alternate" type="text/html" title="Announcing RatZarr, RIOS 2.0.9 and PyShepSeg 2.0.5" /><published>2026-01-30T00:00:00+00:00</published><updated>2026-01-30T00:00:00+00:00</updated><id>https://ubarsc.github.io//update/2026/01/30/ratzarr-1.0.1-rios-2.0.9-pyshepseg-2.0.5</id><content type="html" xml:base="https://ubarsc.github.io//update/2026/01/30/ratzarr-1.0.1-rios-2.0.9-pyshepseg-2.0.5.html"><![CDATA[<h1 id="background">Background</h1>

<p>Although the UBARSC projects rely heavily on the Raster
Attribute Table (RAT) functionality within GDAL, we accept
that for some situations this can be problematic:</p>
<ol>
  <li>RATs with many columns. Eventually this becomes unwieldy.
It would be nice if columns could be grouped in some way
and just access what would need.</li>
  <li>Updating RATs is problematic for files on S3. They must 
be copied locally, updated then copied back. It would be nice
if you could just write a new column to the file on S3.</li>
</ol>

<p>We have identified the <a href="https://zarr.dev/">Zarr Format</a> as 
a useful alternative and are working to support RATs in Zarr
files alongside RATs in normal GDAL files. Among the useful features of
.zarr files, they can be updated on S3 directly.</p>

<h1 id="what-is-changing">What is changing?</h1>

<p>If you are currently using RATs in GDAL files (like KEA and HFA)
and you are happy, then nothing will change.</p>

<h1 id="introducing-ratzarr">Introducing RatZarr</h1>

<p><a href="https://github.com/ubarsc/ratzarr">RatZarr</a> is a new UBARSC project
that creates a RAT-like interface to a Zarr file. Note that it doesn’t
work with any arbitrary .zarr file - only ones created by RatZarr itself. Most
users won’t use this project directly - only via RIOS and/or pyshepseg.
Note that we aren’t talking about saving imagery into a .zarr file -
just the RAT.</p>

<h1 id="rios-209">RIOS 2.0.9</h1>

<p><a href="https://github.com/ubarsc/ratzarr">RIOS</a> 2.0.9 has been released with
support for .zarr files (via RatZarr, if installed) in <a href="https://www.rioshome.org/en/latest/rios_ratapplier.html">ratapplier</a>.
The full list of changes can be <a href="https://www.rioshome.org/en/latest/releasenotes.html">found here</a>.
An example of reading and writing data to/from a .zarr file (alongside
a KEA file) is below:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">rios</span> <span class="kn">import</span> <span class="n">ratapplier</span>
<span class="n">inRats</span> <span class="o">=</span> <span class="n">ratapplier</span><span class="p">.</span><span class="n">RatAssociations</span><span class="p">()</span>
<span class="n">outRats</span> <span class="o">=</span> <span class="n">ratapplier</span><span class="p">.</span><span class="n">RatAssociations</span><span class="p">()</span>

<span class="n">inRats</span><span class="p">.</span><span class="n">vegclass</span> <span class="o">=</span> <span class="n">ratapplier</span><span class="p">.</span><span class="n">RatHandle</span><span class="p">(</span><span class="s">'vegclass.kea'</span><span class="p">)</span>
<span class="n">inRats</span><span class="p">.</span><span class="n">heightclass</span> <span class="o">=</span> <span class="n">ratapplier</span><span class="p">.</span><span class="n">RatZarrHandle</span><span class="p">(</span><span class="s">'heightclass.zarr'</span><span class="p">)</span>
<span class="n">outRats</span><span class="p">.</span><span class="n">vegclass</span> <span class="o">=</span> <span class="n">ratapplier</span><span class="p">.</span><span class="n">RatHandle</span><span class="p">(</span><span class="s">'vegclass.kea'</span><span class="p">)</span>
<span class="n">outRats</span><span class="p">.</span><span class="n">heightclass</span> <span class="o">=</span> <span class="n">ratapplier</span><span class="p">.</span><span class="n">RatZarrHandle</span><span class="p">(</span><span class="s">'heightclass.zarr'</span><span class="p">)</span>

<span class="n">ratapplier</span><span class="p">.</span><span class="nb">apply</span><span class="p">(</span><span class="n">myFunc</span><span class="p">,</span> <span class="n">inRats</span><span class="p">,</span> <span class="n">outRats</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">myFunc</span><span class="p">(</span><span class="n">info</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">outputs</span><span class="p">):</span>
    <span class="n">outputs</span><span class="p">.</span><span class="n">vegclass</span><span class="p">.</span><span class="n">colSum</span> <span class="o">=</span> <span class="n">inputs</span><span class="p">.</span><span class="n">vegclass</span><span class="p">.</span><span class="n">col1</span> <span class="o">+</span> <span class="n">inputs</span><span class="p">.</span><span class="n">vegclass</span><span class="p">.</span><span class="n">col2</span>
    <span class="n">outputs</span><span class="p">.</span><span class="n">heightclass</span><span class="p">.</span><span class="n">colSum</span> <span class="o">=</span> <span class="n">inputs</span><span class="p">.</span><span class="n">heightclass</span><span class="p">.</span><span class="n">col1</span> <span class="o">+</span> <span class="n">inputs</span><span class="p">.</span><span class="n">heightclass</span><span class="p">.</span><span class="n">col2</span>
</code></pre></div></div>

<p>Note that .zarr files can be accessed on S3 using the <code class="language-plaintext highlighter-rouge">s3://bucket/path/to/file.zarr</code>
syntax.</p>

<h1 id="pyshepseg-205">PyShepSeg 2.0.5</h1>

<p><a href="https://www.pyshepseg.org">PyShepSeg</a> 2.0.5 has been released, also with
support for RATs in .zarr files (via RatZarr). The full list of changes 
can be <a href="https://www.pyshepseg.org/en/latest/ReleaseNotes.html">found here</a>.
You would use this functionality if you wanted to save statistics to a .zarr
file, like this:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">pyshepseg</span> <span class="kn">import</span> <span class="n">tiling</span>
<span class="n">segResult</span> <span class="o">=</span> <span class="n">tiling</span><span class="p">.</span><span class="n">calcPerSegmentStatsTiled</span><span class="p">(</span><span class="s">'veginfo.kea'</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> 
    <span class="s">'segment.kea'</span><span class="p">,</span> <span class="n">statsSelection</span><span class="o">=</span><span class="p">[(</span><span class="s">'Band1_Mean'</span><span class="p">,</span> <span class="s">'Mean'</span><span class="p">)],</span>
    <span class="n">outFile</span><span class="o">=</span><span class="s">'s3://bucket/veg.zarr'</span><span class="p">,</span> <span class="n">outFileIsZarr</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
</code></pre></div></div>

<p>You then would presumably do more processing with the data in the .zarr
file with <code class="language-plaintext highlighter-rouge">rios.ratapplier</code> as detailed above.</p>

<h1 id="conclusion">Conclusion</h1>

<p>Reading and writing RAT columns to and from .zarr files is a very 
useful feature, especially for those working in cloud compute 
environment. New releases of RIOS and PyShepSeg allow users to save
their data in .zarr files and any feedback is appreciated. Future
development will include support in TuiView.</p>]]></content><author><name></name></author><category term="update" /><summary type="html"><![CDATA[Background]]></summary></entry><entry><title type="html">Exploring TuiView plugins and developing your own</title><link href="https://ubarsc.github.io//tutorial/2026/01/26/tuiview-plugins.html" rel="alternate" type="text/html" title="Exploring TuiView plugins and developing your own" /><published>2026-01-26T00:00:00+00:00</published><updated>2026-01-26T00:00:00+00:00</updated><id>https://ubarsc.github.io//tutorial/2026/01/26/tuiview-plugins</id><content type="html" xml:base="https://ubarsc.github.io//tutorial/2026/01/26/tuiview-plugins.html"><![CDATA[<h1 id="introduction">Introduction</h1>

<p>TuiView has lots of build-in functionality as <a href="../07/tuiview-intro.html">discussed</a> in
<a href="../16/tuiview-query.html">previous</a> <a href="../21/tuiview-ratquery.html">posts</a>. However,
there are bits of functionality we did not add to TuiView but instead made available
as a plugin. This was either because the use was very obscure and didn’t seem
worth cluttering up TuiView’s codebase, or other dependencies were required and
it didn’t make sense to have TuiView package dependent upon this
other package.</p>

<p>Plugins are a general mechanism that you can use to embed your own 
functionality on top of TuiView. This posts talks about the existing plugins in the
<a href="https://github.com/ubarsc/tuiview-plugins">tuiview-plugins</a> repository before
briefly covering how to develop a plugin yourself.</p>

<h1 id="installing-tuiview-plugins">Installing tuiview-plugins</h1>

<p>The recommended way of installing <code class="language-plaintext highlighter-rouge">tuiview-plugins</code> is from the <a href="https://github.com/ubarsc/tuiview-plugins">git repo</a>:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install </span>git+https://github.com/ubarsc/tuiview-plugins.git
</code></pre></div></div>

<p>Do this into an environment where you already have TuiView installed. To explore
the plugins available, run <code class="language-plaintext highlighter-rouge">tuiviewpluginmgr</code> on the command line. You will be
presented with a Qt based GUI that lists the plugins available with a short
description:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_pluginmgr.png" alt="tuiviewpluginmgr" /></p>

<p>Clicking the “Enabled” check box against one or more plugins will show you how
to set the <code class="language-plaintext highlighter-rouge">TUIVIEW_PLUGINS_PATH</code> environment variable to have it load for different types of shell. 
Note that this environment variable must be set <em>before</em> TuiView is run to get
the plugin to load. 
If you want certain plugin(s) always loaded you can make them Enabled and click
“Save and Exit”. These plugin(s) will be always Enabled when you start <code class="language-plaintext highlighter-rouge">tuiviewpluginmgr</code>.
To ease loading of these plugins each time you run TuiView, you can run:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">eval</span> <span class="sb">`</span>tuiviewpluginmgr <span class="nt">-s</span><span class="sb">`</span>
</code></pre></div></div>
<p>In your <code class="language-plaintext highlighter-rouge">~/.bashrc</code> (for Bash) or your <code class="language-plaintext highlighter-rouge">~/.tcshrc</code> (for tcsh). This will always set
the <code class="language-plaintext highlighter-rouge">TUIVIEW_PLUGINS_PATH</code> for the enabled plugins when you login.</p>

<p>Alternatively, you can copy the relevant plugin files to one of the locations
mentioned in the <a href="https://github.com/ubarsc/tuiview/wiki/Plugins#installing-plugin-file">TuiView Wiki</a>.</p>

<p>When you start TuiView with <code class="language-plaintext highlighter-rouge">TUIVIEW_PLUGINS_PATH</code> set, a message will be printed to the
terminal that lists the plugins that have been loaded. Normally, plugins make a change
to TuiView’s user interface but this depends on how the plugin has been written. Let’s 
look at the various plugins that are part of the <code class="language-plaintext highlighter-rouge">tuiview-plugins</code> repository.</p>

<h1 id="gps-marker">GPS Marker</h1>

<p>This plugin adds a “GPS” menu to the menu bar of each TuiView window. This plugin 
assumes that <a href="https://gpsd.io/">gpsd</a> has been installed on your system and has been
started and that a suitable GPS receiver has been connected to your computer.
The menu has 2 options: “Start Logging” and “Stop Logging”. When you start logging
the current GPS location will be plotted on all TuiView windows:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_gps.png" alt="GPS" /></p>

<p>When the GPS location is updated, then the “bullseye” cursor will move to the new
location on all windows. When you select “Stop Logging” the cursor will be hidden.</p>

<h1 id="timeseries-plot">Timeseries Plot</h1>

<p>This plugin adds a “Timeseries Plot” menu to the menu bar of each TuiView window. The menu has 3 options: 
doing a timeseries on a point, a polygon and how to summarise points in a polygon.
The intention is that you load a stack of images up within one viewer. Ideally, each
file will be annotated with the <code class="language-plaintext highlighter-rouge">LCR_Date</code> metadata item which specifies the date
of the image in <code class="language-plaintext highlighter-rouge">YYYYMMDD</code> format as shown below with <code class="language-plaintext highlighter-rouge">gdal_edit</code>:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gdal_edit <span class="nt">-mo</span> <span class="nv">LCR_Date</span><span class="o">=</span>20250501 image.kea
</code></pre></div></div>

<p>If this is not set, then each image will be assumed to be one day apart, which isn’t
normally what you are after. When <code class="language-plaintext highlighter-rouge">LCR_Date</code> is set, then the number of Julian days
between each image is displayed. To plot a single
point through the timeseries, select “Do a timeseries analysis on a point” from the 
“Timeseries Plot” menu. Then click within the rasters and a timeseries plot will be shown:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_timeseriesplot.png" alt="Timeseries Plot" /></p>

<p>To do a summary of polyon values through a timeseries, select “Do a timeseries analysis on a 
polygon”. How the values within each polygon is summarised for the plot is controlled by the
“Polygon Summary Method” menu option. Note that like other TuiView tools, left click to create a
new vertex of the polygon and right click to close.</p>

<p>Note that like the profile tool, only the displayed bands are shown.</p>

<h1 id="collect-shapefile">Collect Shapefile</h1>

<p>This plugin adds a “Collect” menu to the menu bar. This plugin needs an image loaded
before it will work. The first 3 options on the menu 
(“Create a new Polygon Shapefile”,”Create a new Line Shapefile” and “Create a new Point Shapefile”)
all prompt you for the name of the output shapefile to create. Once you have completed
one of these options, you can select “Collect Feature” from the menu. For polygon and 
line files, left click creates a new vertex and right click finishes the feature. For
points you just need to click within the raster. When you have finished drawing features,
click the “Close Shapefile” option.
The shapefile that has been created will be in the same projection as the loaded
raster file. Each feature will have a unique FID.</p>

<h1 id="scalebar-and-north-arrow">Scalebar and North Arrow</h1>

<p>This plugin adds a “Scale Bar” menu to the menu bar. This plugin can add a scale bar,
a north arrow, a citation and/or a logo to the display in the viewer. The intention is
that any time you save the current display, then these extra things will also be
saved to the .png making the result something that could be put into a report or
some other document where this extra information would be useful. To enable a 
scale bar, select the “Show Scale Bar” menu option. To enable a north arrow, 
select the “Show North Arrow” menu option. To add citation text, select “Set
citation text” and you will be prompted for the text to display. To add a logo,
select “Set logo” and you will be prompted for an image file to display as the logo.
Here is an example of a scale bar, north arrow, citation and logo all displayed
within a TuiView window:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_scalebar.png" alt="Scale Bar" /></p>

<h1 id="qml-reader">QML Reader</h1>

<p>For compatibility with QGIS, the QML Reader plugin allows .qml files to be 
read and the colour information applied to a single band image via the 
stretch window. This plugin just adds one button to the stretch window:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_qml.png" alt="QML" /></p>

<p>Note how there is an extra button (<img src="https://raw.githubusercontent.com/ubarsc/tuiview-plugins/refs/heads/master/tuiview_plugins/qml_reader/qgis_qml_icon.svg" alt="QML" width="32" height="32" />) on the right hand side of the toolbar
to allow loading of the QML file. When you press this button, you will
be prompted for the QML file to apply. Once you have done this the .qml
file will be applied to the the loaded image and the stretch window will
be closed. Note that the image must be single band and that the Stretch 
type will automatically be set to None before the QML is applied to match
QGIS. Note also that altering the stretch parameters and applying the 
change will lose the colouring information in the window - you have to set the
colours via the QML button.</p>

<h1 id="location-broadcast">Location Broadcast</h1>

<p>This plugin does not make any change to the user interface. It logs to a 
file each time that the TuiView windows are zoomed or panned. This file
is saved under the <code class="language-plaintext highlighter-rouge">/tmp</code> directory, or whatever your <code class="language-plaintext highlighter-rouge">TMP</code> environment
variable is set to. The filename will be <code class="language-plaintext highlighter-rouge">locationbcast_XXXX</code> where <code class="language-plaintext highlighter-rouge">XXXX</code>
is your user ID (or username under Windows). Each time TuiView moves
or pans, this file is rewritten with a timestamp and the new bounds.
This idea is that other software can monitor this file and update as 
needed when the location moves.</p>

<h1 id="recode">Recode</h1>

<p>This plugin creates a “Recode” menu on the menu bar. This plugin is for
manually editing rasters by drawing polygons. Instead of updating the raster directly, it
saves each polygon and recode rules for later editing or creation of new
image with these edits applied.</p>

<p>To recode an image, load a thematic image into TuiView. Click “Start Recoding
Top Layer” under the “Recode” menu. To recode values in an area, click
“Recode Polygon” and select a polygon in the usual TuiView way (left click
for new vertex, right click closes). Then you will be presented with a table
of recodes to be made:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_recode.png" alt="Recode" /></p>

<p>The left column shows the value in the image, the right column (which can be
edited by double clcking) shows the value you wish to change it to. Many recodes
can happen in one polygon in this way. There is space for a comment that applies
to this polygon below the table. When you click OK then the recode will be applied
to the current image so you can see the new values. However the original raster
will not be updated. You can visually see the outlines of your recode polygons
by clicking “Show Oulines of Polygons”. To change the recode rules for a polygon,
select “Edit Recodes of a Polygon” in the “Recode” menu. You can then click a 
point within a polygon and the recode table will again be shown and you can make
changes as appropriate. When you want the updates saved, click “Save recodes to file”
in the “Recode” menu. This will save a JSON file with all the polygons and recodes
the same as the raster file but with <code class="language-plaintext highlighter-rouge">.recode</code> appended onto the file name. So, for
example, if you were editing <code class="language-plaintext highlighter-rouge">/tmp/a.kea</code> then your edits will be saved as <code class="language-plaintext highlighter-rouge">/tmp/a.kea.recode</code>.</p>

<p>If you were to start a new TuiView and start recoding the same image again you will
be asked whether you want to load the existing recodes and add to them, or start a
new set of recodes.</p>

<p>To create a new raster file from the original raster file and edits stored in a <code class="language-plaintext highlighter-rouge">.recode</code> 
file, use the <code class="language-plaintext highlighter-rouge">newfile_from_recode</code> entry point that was created when you
installed <code class="language-plaintext highlighter-rouge">tuiview-plugins</code>.</p>

<h1 id="creating-your-own-plugins">Creating your own plugins</h1>

<p>It is relatively easy to create your own plugins. As described in the
<a href="https://github.com/ubarsc/tuiview/wiki/Plugins">plugin wiki page</a>, you need
to create a Python file with four functions plus an event handler if 
your plugin needs to respond to events. Because TuiView looks at all Python
files in the directory(s) set in <code class="language-plaintext highlighter-rouge">TUIVIEW_PLUGINS_PATH</code>, we recommend starting 
by creating a directory with just your plugin file in it:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir test
export </span><span class="nv">TUIVIEW_PLUGINS_PATH</span><span class="o">=</span><span class="sb">`</span><span class="nb">pwd</span><span class="sb">`</span>/test
<span class="c"># edit test/plugin.py</span>
</code></pre></div></div>

<p>Below is a very simple plugin that adds a menu to the menu bar and
displays a message box when the user selects “My plugin Action”:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_plugintest.png" alt="Plugin Test" /></p>

<p>The source code is shown below:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">PySide6.QtCore</span> <span class="kn">import</span> <span class="n">QObject</span>
<span class="kn">from</span> <span class="nn">PySide6.QtGui</span> <span class="kn">import</span> <span class="n">QAction</span>
<span class="kn">from</span> <span class="nn">PySide6.QtWidgets</span> <span class="kn">import</span> <span class="n">QMessageBox</span>
<span class="kn">from</span> <span class="nn">tuiview</span> <span class="kn">import</span> <span class="n">pluginmanager</span>

<span class="k">def</span> <span class="nf">name</span><span class="p">():</span>
    <span class="k">return</span> <span class="s">'Test'</span>

<span class="k">def</span> <span class="nf">author</span><span class="p">():</span>
    <span class="k">return</span> <span class="s">'Sam Gillingham'</span>

<span class="k">def</span> <span class="nf">description</span><span class="p">():</span>
    <span class="k">return</span> <span class="s">'Tests that plugin works'</span>
    
<span class="k">class</span> <span class="nc">MyEventHandler</span><span class="p">(</span><span class="n">QObject</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">viewer</span><span class="p">):</span>
        <span class="n">QObject</span><span class="p">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">viewer</span> <span class="o">=</span> <span class="n">viewer</span>

    <span class="c1"># this function gets called when action triggered
</span>    <span class="k">def</span> <span class="nf">myEvent</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="n">QMessageBox</span><span class="p">.</span><span class="n">information</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">viewer</span><span class="p">,</span> <span class="s">"Viewer"</span><span class="p">,</span> <span class="s">"My Plugin"</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">action</span><span class="p">(</span><span class="n">actioncode</span><span class="p">,</span> <span class="n">viewer</span><span class="p">):</span>
    <span class="c1"># check for the code we are interested in 
</span>    <span class="k">if</span> <span class="n">actioncode</span> <span class="o">==</span> <span class="n">pluginmanager</span><span class="p">.</span><span class="n">PLUGIN_ACTION_NEWVIEWER</span><span class="p">:</span>

        <span class="c1"># create a handler class
</span>        <span class="n">handler</span> <span class="o">=</span> <span class="n">MyEventHandler</span><span class="p">(</span><span class="n">viewer</span><span class="p">)</span>
        <span class="c1"># create an action class
</span>        <span class="n">myaction</span> <span class="o">=</span> <span class="n">QAction</span><span class="p">(</span><span class="n">viewer</span><span class="p">,</span> <span class="n">triggered</span><span class="o">=</span><span class="n">handler</span><span class="p">.</span><span class="n">myEvent</span><span class="p">)</span>
        <span class="n">myaction</span><span class="p">.</span><span class="n">setText</span><span class="p">(</span><span class="s">"My plugin Action"</span><span class="p">)</span>

        <span class="c1"># create a new menu and install the action
</span>        <span class="n">mymenu</span> <span class="o">=</span> <span class="n">viewer</span><span class="p">.</span><span class="n">menuBar</span><span class="p">().</span><span class="n">addMenu</span><span class="p">(</span><span class="s">"&amp;My Menu"</span><span class="p">)</span>
        <span class="n">mymenu</span><span class="p">.</span><span class="n">addAction</span><span class="p">(</span><span class="n">myaction</span><span class="p">)</span>

        <span class="c1"># make sure the object isn't garbage collected
</span>        <span class="n">viewer</span><span class="p">.</span><span class="n">plugins</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
</code></pre></div></div>

<p>Obviously, some experience with Qt is required for building plugins. You need to
create a <code class="language-plaintext highlighter-rouge">QObject</code> derived class to connect to signals and handle them. This class
must be added to the <code class="language-plaintext highlighter-rouge">plugins</code> list of the window it belongs to so it does not
get garbage collected by Python. You also need to be aware how TuiView works 
behind the scenes. The existing plugins are useful for this and there is some
<a href="https://tuiview.readthedocs.io/en/latest/">developer documentation</a> available
but the TuiView source code is the best reference.</p>

<h1 id="conclusion">Conclusion</h1>

<p>The existing plugins in the <a href="https://github.com/ubarsc/tuiview-plugins">tuiview-plugins repo</a>
provide some useful extensions to TuiView functionality. There is plenty of scope
for writing specific TuiView plugins to address specific functionality that may
just be required for a particular workflow or company.</p>]]></content><author><name></name></author><category term="tutorial" /><summary type="html"><![CDATA[Introduction]]></summary></entry><entry><title type="html">Querying Raster Layers with Raster Attribute Tables</title><link href="https://ubarsc.github.io//tutorial/2026/01/21/tuiview-ratquery.html" rel="alternate" type="text/html" title="Querying Raster Layers with Raster Attribute Tables" /><published>2026-01-21T00:00:00+00:00</published><updated>2026-01-21T00:00:00+00:00</updated><id>https://ubarsc.github.io//tutorial/2026/01/21/tuiview-ratquery</id><content type="html" xml:base="https://ubarsc.github.io//tutorial/2026/01/21/tuiview-ratquery.html"><![CDATA[<h1 id="introduction">Introduction</h1>

<p>In a <a href="../16/tuiview-query.html">previous post</a> we looked at querying
continuous rasters with TuiView. In this post we look at querying
thematic rasters in TuiView. TuiView treats any raster with an 
attribute table as thematic.</p>

<p>Outputs from <a href="https://www.pyshepseg.org/en/latest/">pyshepseg</a> 
generally contain raster attribute tables. See our 
<a href="../../../2025/12/11/pyshepseg-intro.html">previous post</a> on 
performing a segmentation and gathering statistics that are
put into the raster attribute table.</p>

<p>KEA and HFA are the only GDAL drivers that currently properly support
raster attribute tables (RAT) natively. For some drivers (like GeoTiff),
GDAL saves the RATs in a sidecar .xml file. This can be very inefficient
and slow for large RATs so we recommend using KEA or HFA where possible.
Some drivers (again, like GeoTiff) support a similar idea, but just for colors,
called “Colour Tables”.</p>

<p>Raster attribute tables can have colour columns. If the RAT has 
colour tables, or there is a Colour Table present, you can ask that TuiView 
open the file as “Color Table” and you will see the behaviour described in this 
post. This is the default behaviour of TuiView
but can be overridden in the the stretch dialog, or in the default stretch.</p>

<h1 id="querying-thematic-rasters">Querying Thematic Rasters</h1>

<p>If you open a file that has a Raster Attribute Table with colours, you will
see something like the following when you press the Query Button (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/query.png" alt="Query Window" />):</p>

<p><img src="https://ubarsc.github.io//images/tuiview_ratquery.png" alt="Query Window" /></p>

<p>Note that the image is displayed using the colours in the colour table. Also,
note how that the color columns in the RAT are also shown as a single
“Color” column with the actual colour shown as a rectangle. The other
columns of the RAT are shown with their values. The rows are also numbered
on the left and the table can be scrolled up and down.</p>

<p>Only the currently shown part of the table is loaded into memory. This means
that TuiView can handle enormous RATs without running out of memory.</p>

<h1 id="highlighting-a-row-in-the-table-and-highlighting-areas-of-the-raster-by-selecting-rows">Highlighting a row in the table and highlighting areas of the raster by selecting rows</h1>

<p>If you click on the raster with the Query Tool, the corresponding row will 
be highlighted in yellow and the table will be scrolled to 
show that row. 
If you click on a row it is selected. TuiView makes a distinction between
highlighted and selected rows. Selected rows are shown in blue in the
table. You can select more than one row by holding down the Ctrl key before
clicking another row - the first one will stay selected. You can add as
many rows as you like to the selection using this method. If you want to select
a range of rows, click the first row of the range then while holding the Shift
key, click the last row. The rows you clicked on will be selected along
with the rows between.
By default, the areas on the map that correspond to the selected row(s) will
be highlighted in yellow. You can prevent the map being highlighted using the
Highlight Selection button <img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/highlight.png" alt="Highlight Selection" /> 
on the toolbar. Next to this button is a yellow button that allows you to change
the selection colour. Clicking this brings up a colour chooser and when 
you select a colour, selected rows are shown as this colour on the map and the 
colour of this button on the toolbar is updated.
There are also buttons to remove all selected rows (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/removeselection.png" alt="Remove Selection" />)
and select all rows (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/selectall.png" alt="Select All" />).</p>

<p>This “Faster Forward” and “Rewind” Buttons can be used to scroll up or down to the
next selected row(s) in the table.</p>

<h1 id="changing-how-the-table-is-displayed">Changing how the table is displayed</h1>

<p>By default, the columns appear in the order they were written to the file
(with the exception that the colour column appears first). If you wish to change
the order in which the columns appear, right click on a column header and select
“Move Left”, Move Right”, “Move Left Most” or “Move Right Most”. The column
will then move as directed. You can save the column order so TuiView uses
this next time you open the file by selecting the Save Column Order Button
(<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/savecolumnorder.png" alt="Save Column Order Button" />)
but the file must be in update mode - more detail on update mode is 
provided below.
If you right click on a floating point column, an option called “Set
number of decimal places” appears. Use this to change the number of
decimal places from the default (2).</p>

<h1 id="geographical-selection">Geographical Selection</h1>

<p>You can also select rows by selecting areas on the map. There are
3 ways to do this: by polygon, by line and by point.
To select rows by polygon, select the Geographic Selection by Polygon Tool
(<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/geographicselect.png" alt="Polygon" />).
Then on the map, left click the first vertex of the polygon, then the next and the next.
When you have finished, right click and the polygon will be closed. The rows
that intersect with this polygon will be highlighted on the map and selected
in the table:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_geogselect.png" alt="Geographic Select" /></p>

<p>You can also select rows that sit along a line using the Geographic Select
by Line Tool (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/geographiclineselect.png" alt="Line" />).
This operates similiarly to the Profile Tool. Left click on the start of the line, and
any vertices. Right click to end the line. Rows that intersect the line will
be highlighted on the map and selected in the table.</p>

<p>Lastly, you can select points using the Geographic Select by Point Tool
(<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/geographicpointselect.png" alt="Point" />).
Simply click a point on the map and the row for that point will be selected.</p>

<p>Note that with all these tools, holding the Ctrl key down will add the
new rows to the current selection instead of unselecting any rows first.</p>

<h1 id="select-by-expression">Select by expression</h1>

<p>There is another way of selecting rows in the RAT - with an expression.
To select by expression, select the Select using an Expression button
(<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/userexpression.png" alt="Select using an Expression" />).
A window is opened that allows you to enter an expression (plus any imports your expression may require).
Each column appears as a Python variable (actually a numpy array). There are some tricks about combining
expressions and “extra” columns provided in the help text. Entering an expression that returns a boolean
and pressing “Apply”
selects the matching rows in the table (and highlights on the map):</p>

<p><img src="https://ubarsc.github.io//images/tuiview_expr.png" alt="Select by expression" /></p>

<p>Note this window stays open until you select “Close” so you can easily modify your expression.
Also, be aware that your expression is evaluated multiple times, once for each “block”
of the RAT (in a manner similar to <code class="language-plaintext highlighter-rouge">rios.ratapplier</code>) so you will need to ensure that anything
returned matches the shape of the input(s) to your expression. For example, to select 
random rows, you may do something like this:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">numpy</span><span class="p">.</span><span class="n">random</span><span class="p">.</span><span class="n">rand</span><span class="p">(</span><span class="o">*</span><span class="n">row</span><span class="p">.</span><span class="n">shape</span><span class="p">)</span> <span class="o">&lt;</span> <span class="mf">0.5</span>
</code></pre></div></div>

<p>This uses the shape of the <code class="language-plaintext highlighter-rouge">row</code> array to create a boolean array of the correct shape.</p>

<h1 id="saving-selected-rows-as-a-csv">Saving selected rows as a .csv</h1>

<p>Once you have selected rows, either manually, by geographic select or by expression
(or a mix of all 3) you can save the selected rows by using the Export Selected Rows
to CSV button on the toolbar (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/csv.png" alt="Export CSV" />).
You are prompted for the name of the file to save.
This means the data from the selected rows can be imported into a spreadsheet, a database or
some other custom Python script for further processing.</p>

<h1 id="updating-the-rat">Updating the RAT</h1>

<p>The last main feature that TuiView has when viewing a thematic raster file is the ability
to update the file. To do this you need to put the file into “Update” mode. This is done
by using the Toggle updates to dataset button on the toolbar (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/lock.png" alt="" />).
While the file is open in update mode, this button stays down. To flush all changes to the 
file, press this button again and the file will go back to being open in read only mode.</p>

<p>Once the file is open in update mode, the following features become available:</p>
<ol>
  <li>The column order can be saved to the file (changing the column order is discussed above)
by using the Save Column Order button (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/savecolumnorder.png" alt="Save Column Order" />) on the toolbar.
Note that the new order is only visible by TuiView and it won’t change what is seen
in other software.</li>
  <li>New columns can be created using the Add Column button (<img src="https://raw.githubusercontent.com/ubarsc/tuiview/refs/heads/master/resources/addcolumn.png" alt="Add Column" />) on the toolbar.
You will be prompted for the type and name of the column</li>
  <li>You can edit selected rows of a column using an expression.</li>
  <li>You can updated selected rows of a column with a value entered on the keyboard.</li>
</ol>

<p>These last two features are discussed below.</p>

<p>To update selected rows of a column, right click on the column you wish to 
modify. Select “Edit Selected Rows in Column” option. You will then be presented with
a similar window to the “Select by Expression” case, but here your expression
needs to yield a value to put in the array. This can be done by using other
columns (again available as numpy arrays) or a scalar:</p>

<p><img src="https://ubarsc.github.io//images/tuiview_updateexpr.png" alt="Geographic Select" /></p>

<p>Pressing “Apply” will update the column for selected rows. Note that like
selecting rows by expression, the expression entered here will be applied
to the RAT in blocks.</p>

<p>For quickly updating selected rows in a column, there is another option
for setting values from the keyboard. To use this, right click on the 
column you wish to update and select the “Set column to receive keyboard edits”
option in the menu. Select the rows you wish to update, enter the values with
the keyboard and press enter. The selected rows will then be updated. To
get out of this mode, right click on the column and select “Set column to 
receive keyboard edits” again which should uncheck the tickbox next to this 
option. 
This option is designed for quick classification of segments. The idea is 
that you select your row(s) with a geographic selection and quickly update
these to a value that you enter with the keyboard, and then move onto 
the next segment(s). The resulting column can be used to train a model.</p>

<p>It is worth noting that you can edit the colours by either updating 
the individual colour columns or right clicking on the “Color” column
and selecting “Set Color of Selcted Rows”. This will bring up a colour chooser
where you can choose a colour and the colour columns will be updated.</p>

<h1 id="conclusion">Conclusion</h1>

<p>TuiView has some sophisticated tools for querying and updating thematic
rasters. These features go beyond what most packages provide and make
working with Raster Attribute Tables a powerful tool.</p>]]></content><author><name></name></author><category term="tutorial" /><summary type="html"><![CDATA[Introduction]]></summary></entry></feed>