SEPIA
Software documentation for the APEX SEPIA receivers
The SEPIA receiver control for APEX

IMPORTANT This documentation describes the structure and inner workings of the sepia program, as well as how to compile, install and update the software. A user manual is available from within the sepia program via its Help menu.

About the program

Log into sepia.apex-telescope.org (10.0.2.147) as user apex. The official executable is installed in /usr/local/bin, whereas the source code is in directory /home/apex/SEPIA, see section for building a new version.

Introduction

The control program for the SEPIA receiver channels on APEX is built using Qt, a cross-platform application and UI framework for developers using C++. Although the control of the receiver should work entirely via commands sent from the APECS system, it does come with a graphical user interface which provides some functionality in order to quickly check system status and to perform some fundamental operations (like startup and shutdown) manually.

The control software connects to the following hardware devices:

Device connection protocol comment class
APECS system ethernet udp SCPI ScpiRequest
M&C unit USB CANbus following ALMA standards MCunit
GP I/O ethernet tcp Modbus Acromag
IF-switch ethernet tcp Mini-Circuit MiniCircuits
Backend switch ethernet tcp Mini-Circuit MiniCircuits
Anritsu synth ethernet tcp SCPI Synthesizer
Selection mirrorethernet tcp via Lantronics to RS-232 SelectMirror

The monitoring of the vacuum pressure and temperature, as well as the status of the UPS unit, is done by a separate process using the following devices:

Device connection protocol comment class
Cryo controller ethernet udp CryoController
UPS ethernet snmp tbd UPSunit
Vacuum monitor ethernet tcp via Lantronics to RS-232 VacuumController

The configuration of the network is outlined in file network.h

Updating the software to the latest version

This section assumes that git has been setup and configured properly (see installing on a new machine). 

$ cd ~/SEPIA
$ git pull
$ make clean
$ make
$ sudo make install

This documentation and the user manual are generated by running 

$ make documentation
$ make help

To install configuration files in ~/.config/APEX/ after an update, run 

$ make config

VNC setup

The apex user is automatically logged in after reboot of the computer. A vncserver is started via systemd on port 5901 to allow access to the desktop of the apex user via sepia:1 (passsword: apecs4op). The port 5902 (i.e. vnc to sepia:2) is forwarded to the GARD laptop at address 192.168.1.10 on the internal sepia LAN.

Installing on a new machine

The following part of the documentation is only necessary in case the control computer is being replaced by a new machine.

The installation is based on a Debian-9 64-bit system (code name "stretch"), i.e. a Linux distribution which will be supported as an LTS release until June 2022, but other Debian based distributions (e.g. Ubuntu, Mint) should work as well.

The installation should include the creation of a user apex with sudo privileges, such that all steps below can be performed by this user.

After the basic install, which includes a graphical user environment based on the lxqt desktop, the following packages (and its dependencies) were added: 

$ # support for building the usb CANbus kernel module
$ sudo apt-get install -y bzip2 build-essential libncurses5-dev libelf-dev libpopt-dev
$ # use systemd as a replacement for sysvinit
$ sudo apt-get install -y systemd-sysv
$ # git version control, emacs editor, secure shell and debugger
$ sudo apt-get install -y git emacs ssh openssh-server gdb
$ # bison and fles to build the SCPI parser used by sepia
$ sudo apt-get install -y bison flex
$ # tools to generate documentation from source code
$ sudo apt-get install -y doxygen graphviz
$ # Qt5 development and qwt plotting library
$ sudo apt-get install -y qt5-default libqwt-qt5-6 libqwt-qt5-dev
$ sudo apt-get install -y libqt5svg5-dev libqt5opengl5-dev libqt5scripttools5 qtscript5-dev
$ # Postgres SQL database
$ sudo apt-get install -y postgresql
$ # support for Sqlite databases
$ sudo apt-get install -y sqlite3 sqlitebrowser libqt5sql5-sqlite
$ # install vncserver
$ sudo apt-get install -y vnc4server
$ # optional: allow network access to local shares via SMB protocol
$ sudo apt-get install -y samba samba-client

Set up the Postgres SQL database

$ sudo su postgres                 # become user postgres
$ createdb sepia
$ createuser -d -P apex
$ psql -d sepia
psql (9.6.9)
Type "help" for help.
sepia=# GRANT all ON DATABASE sepia TO apex;
sepia=# \q                         # exit psql
$ exit                             # become user apex again

Create tables and populate all tuning tables: 

$ cd ~/SEPIA/SQL
$ psql -U apex -d sepia
psql (9.6.9)
Type "help" for help.
sepia=> \i sepia.sql
sepia=> \q                         # exit psql

Once the postgres database is set up, it is accessible from anywhere in the APEX network (10.0.0.0/16). E.g. when using the postgresql client (psql) you may use the following: 

$ psql -U apex -h sepia -d sepia   # password apecs4op

The -h switch is only necessary if you connect from another machine. You will find the following tables 

psql (9.6.10)
Type "help" for help.

sepia=> \dt
           List of relations
 Schema |     Name      | Type  | Owner
--------+---------------+-------+-------
 public | sepia180log   | table | apex
 public | sepia180table | table | apex
 public | sepia345log   | table | apex
 public | sepia345table | table | apex
 public | sepia660log   | table | apex
 public | sepia660table | table | apex
 public | sepiaenv      | table | apex
(7 rows)
  1. sepiaenv is filled by the sepialogger service with readings every minute of the cartridge temperature, pressure and UPS status. It will also tell you if the temperature control loop of the 4K stage is switched on and, if yes, to what temperature the control loop is programmed.
  2. sepiaXXXlog will contain a log of all tunings for each receiver, where XXX is one of 180, 345 or 660. These tables have the same structure as the ones that were used for SHeFI and the old band 5 and band 9 systems.
  3. sepiaXXXtable are the tuning tables, where XXX again is 180, 345 or 660.

Tables 1. and 2. should be regarded as read-only, as they are filled by the sepia software. Changes to the tuning tables should also preferably not be made by SQL statements directly, but rather by preparing SQL scripts which can be put under version control.

Installing the CANbus driver and API

The control software uses a PEAK CANbus usb-interface to communicate with the M&C unit (http://www.peak-system.com/PCAN-USB.199.0.html).

The following software has to be downloaded:

http://www.peak-system.com/fileadmin/media/linux/files/peak-linux-driver-8.x.x.tar.gz

where 8.x.x stands for the latest version number available. Unpack this, then compile and build it as follows: 

$ cd peak-linux-driver-8.6.0
$ make clean
$ make NET=NO # to build the PEAK chardev interface.
$ sudo make install

The API comes in the form of a shared library:

http://www.peak-system.com/produktcd/Develop/PC%20interfaces/Linux/PCAN-Basic_API_for_Linux/PCAN_Basic_Linux-2.x.x.tar.gz

Again, unpack this, then compile and build it as follows: 

$ cd PCAN_Basic_Linux-2.x.x/pcanbasic
$ make
$ sudo make install

Building the control software from scratch

Assuming that access to the git repository at git@nain.oso.chalmers.se has been set up.

Initial cloning of the repository: 

$ cd ~
$ # configure your git setup if necessary
$ git config --global user.email "apex@sepia"
$ git config --global user.name "APEX"
$ git clone git@nain.oso.chalmers.se:/home/git/SEPIA.git
$ cd SEPIA
$ qmake sepia.pro
$ make parser
$ make help           # produces inline user manual
$ make
$ sudo make install
$ make documentation  # uses this file to generate html pages

Once the SEPIA folder exists and you want to make sure you are on the latest version:

Build SEPIA logger and viewer

First, build the environement logger and viewer and install in /usr/local/bin 

$ cd ~/SEPIA/Environment
$ qmake environment.pro
$ sudo make install

This will build both sepialogger and sepiaviewer (a small graphical application which allows you to look at the latest readings from the logger, when the sepia itself is not running). It will also install sepialogger as a system service, as well as install the DBus interface used by the sepiaviewer and the sepia GUI applications. 

$ sudo systemctl enable sepialogger.service   # for start at next reboot
$ sudo systemctl start sepialogger.service    # for immediate start
$ systemctl status sepialogger.service

Once the sepialogger has started to produce data in database table sepiaenv, the following python3 script can be used to produce plots of temperatures and pressure versus time: 

#!/usr/bin/python3

import psycopg2
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
from matplotlib import dates

conn = psycopg2.connect(host="sepia",user="apex",password="apecs4op",database="sepia")

sql = "SELECT * FROM sepiaenv ORDER BY tstamp"
sql = "SELECT * FROM sepiaenv WHERE tstamp > '2018-08-14' ORDER BY tstamp"
tbl = pd.read_sql(sql, conn)

t = pd.to_datetime(tbl['tstamp'], format="%Y-%m-%d %H:%M:%S")
t = np.array(t)

Ta = tbl['channela']
Tb = tbl['channelb']
Tc = tbl['channelc']
Td = tbl['channeld']
p  = tbl['pressure']*1.0e6

hfmt = dates.DateFormatter("%d-%b %Hh")
fig = plt.figure()
plt.suptitle("SEPIA cryo data\n" + sql)
# plt.suptitle(sql)

ax = fig.add_subplot(5,1,1)
ax.tick_params(labelbottom=False)
plt.plot(t, Ta, 'b-')
plt.ylabel("channel A [K]")
plt.grid(axis="y", linewidth=1, color="grey", linestyle="dotted")

ax = fig.add_subplot(5,1,2)
ax.tick_params(labelbottom=False)
plt.plot(t, Tb, 'b-')
plt.ylabel("channel B [K]")
plt.grid(axis="y", linewidth=1, color="grey", linestyle="dotted")

ax = fig.add_subplot(5,1,3)
ax.tick_params(labelbottom=False)
plt.plot(t, Tc, 'b-')
plt.ylabel("channel C [K]")
plt.grid(axis="y", linewidth=1, color="grey", linestyle="dotted")

ax = fig.add_subplot(5,1,4)
ax.tick_params(labelbottom=False)
plt.plot(t, Td, 'b-')
plt.ylabel("channel D [K]")
plt.grid(axis="y", linewidth=1, color="grey", linestyle="dotted")

ax = fig.add_subplot(5,1,5)
ax.xaxis.set_major_formatter(hfmt)
plt.plot(t, p, 'r-')
plt.ylabel("pressure [nbar]")
plt.grid(axis="y", linewidth=1, color="grey", linestyle="dotted")
plt.xticks(rotation=90)

plt.show()

Necessary initialization files

The files SEPIA.conf and CANbus.ini need to be copied to ~/.config/APEX/

  • file SEPIA.conf specifies default values to be used for LNA drain voltages and currents, default coil currents and mixer bias and current offsets.
  • file CANbus.ini describes the CANbus address mapping.

Tuning table database

The program expects the tuning table to be available in a Postgresql database called 'sepia', table 'sepia180table', 'sepia345table' and 'sepia660table'. This is the default. Alternatively, the program may be recompiled to use sqlite instead, simply add a line DEFINES += SQLITE in the sepia.pro project file. The program will then expect a sqlite database in file ~/.config/APEX/sepia.db. To have a look at the table, either start the sepia program and access the tables via the 'Tune -> Tuning tables' menu, or use the 'psql' utility from the command line: 

psql -U apex -d sepia
Password for user apex:    (enter the password for the apex account)
psql (9.6.9)
Type "help" for help.
sepia=> select * from sepia180table; -- in order to see band 5 table
sepia=> select * from sepia345table; -- in order to see band 7 table
sepia=> select * from sepia660table; -- in order to see band 9 table

Configure VNC server as a system service

Copy the following to /etc/systemd/system/vncserver.service 

[Unit]
Description=Remote desktop service (VNC)
After=syslog.target network.target

[Service]
Type=forking
User=apex
PAMName=login
PIDFile=/home/apex/.vnc/%H:1.pid
ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill :1 > /dev/null 2>&1 || :'
ExecStart=/usr/bin/vncserver -geometry 1280x960 -depth 24 :1
ExecStop=/usr/bin/vncserver -kill :1

[Install]
WantedBy=multi-user.target

Enable and start the service: 

$ sudo systemctl enable vncserver.service   # for start at next reboot
$ sudo systemctl start vncserver.service    # for immediate start
$ systemctl status vncserver.service

The SCPI interface and state machine implementation

The typical sequence per scan looks like this (SEPIAxxx should be replaced with one of SEPIA180, SEPIA345 or SEPIA660). In case the SEPIA channel was not used during the previous scan, the channel will be selected via 

APEX:SEPIAxxx:LO1:on

and deselected via 

APEX:SEPIAxxx:LO1:off

For all scans: 

APEX:SEPIAxxx:cmdSideBand <sideband>
APEX:SEPIAxxx:cmdSkyFrequency <frequency [GHz]>
APEX:SEPIAxxx:cmdFreqSwitchMode ON | OFF
APEX:SEPIAxxx:cmdFreqSwitchThrows <low offset [MHz]> <high offset [MHz]> # 0 0 for non-FSW modes
APEX:SEPIAxxx:tune

Monitoring items are polled once per minute. Queries of properties which are note supported by SEPIAxxx will generate an ERROR: UNKNOWN_PROPERTY response. 

APEX:SEPIAxxx:tuningModified?

will result in YES if the commanded LO tuning has changed by more than the tuning range of the PLL, whereas smaller changes due to Doppler corrections, which the PLL can handle without losing lock, will generate NO.

States

  • Initial state (after band has been selected via GUI): SHUTDOWN
  • a APEX:SEPIAxxx:on will result in state DISABLED via INITIALIZE, the same is achieved via the initialize command of the GUI.
  • APEX:SEPIAxxx:LO1:on will go from DISABLED to ENABLED
  • APEX:SEPIAxxx:LO1:off will go from ENABLED to DISABLED
  • APEX:SEPIAxxx:off will go from DISABLED to SHUTDOWN
  • any error will result in state FAULTED
  • APEX:SEPIAxxx:reset will restore DISABLED in case sepia can recover from the error (like a lost frequency lock).

Typically, the APEX:SEPIAxxx:on/off commands are not sent by APECS, this is typically handled via the GUI instead. Currently, APECS only sends the

When in state FAULTED, any tune command and all updates of commanded properties will generate an ERROR. This will abort the scan and the observer will receive a message via the observing log.

State machine