Skip to content

Set up your first experiment

This page walks through creating a midas experiment from scratch, taking some test data, and making the experiment start automatically. Everything runs on a single machine (the "experiment host"). Once this is working, see running on multiple machines to add programs running on other computers.

It assumes you have already installed midas, and set up your environment so that $MIDASSYS is defined and the midas programs are in your $PATH.

Used the install script?

The install script has already done the first few steps for you: it set up your environment, created an experiment directory and ODB, and started mhttpd, mlogger and an example frontend. You can skip to taking a first run, but note that the install script's mhttpd listens on port 8081 rather than 8080.

The examples on this page use an experiment called test_expt, in the directory /home/daq_user/online, with midas installed in /home/daq_user/packages/midas. Substitute your own names and paths.

Setup

Creating the experiment directory

Midas supports having multiple experiments on the same machine, each with their own experiment directory. The experiment directory holds the ODB, the message log, and (by default) any data files that get written.

mkdir /home/daq_user/online
cd /home/daq_user/online

Defining the experiment

Midas programs need to know where your experiment directory is. There are two ways to tell them, and most experiments use an exptab file. Either way, you set some environment variables in your shell's startup file (.bashrc, .cshrc or .zshrc), alongside MIDASSYS and PATH.

An "exptab" file lists the name and directory of each experiment on the machine. It has one line per experiment, containing the experiment name, the experiment directory, and the username of the "owner" of that experiment. Each field should be separated by one or more spaces, and lines starting with # are ignored.

Example exptab

#Expt name     Expt directory               Username
test_expt      /home/daq_user/online        daq_user
decay          /home/daq_user/decay_daq     daq_user
e512           /home/other_user/expt_512    other_user

You can create this file with any text editor, or have odbinit create it for you by running odbinit --exptab -e test_expt.

Then set these environment variables:

Variable Value
MIDAS_EXPTAB The location of your exptab file. If not set, midas looks for /etc/exptab and /exptab
MIDAS_EXPT_NAME Optional. The experiment to use, if your exptab lists more than one. Can be overridden by the -e command-line flag 
export MIDAS_EXPTAB=/home/daq_user/online/exptab
export MIDAS_EXPT_NAME=test_expt

setenv MIDAS_EXPTAB /home/daq_user/online/exptab
setenv MIDAS_EXPT_NAME test_expt

Generating an environment script with odbinit

Instead of editing your startup file, you can run odbinit --env in the experiment directory. This writes env.sh (for bash and zsh) and env.csh (for csh), which set MIDASSYS, PATH and MIDAS_EXPTAB (and MIDAS_SERVER_HOST when used on a different machine). Load it with source /home/daq_user/online/env.sh, and add that line to your startup file.

If you only ever have one experiment, you can skip the exptab file, and set the $MIDAS_DIR environment variable to the experiment directory instead. You must also set $MIDAS_EXPT_NAME to the name of the experiment.

export MIDAS_DIR=/home/daq_user/online
export MIDAS_EXPT_NAME=test_expt

setenv MIDAS_DIR /home/daq_user/online
setenv MIDAS_EXPT_NAME test_expt

MIDAS_DIR overrides MIDAS_EXPTAB, so don't set both.

Then open a new terminal, or run source ~/.bashrc (or the equivalent for your shell). The rest of this page uses the exptab approach, but everything works the same with MIDAS_DIR.

Programs running on other computers connect differently, using MIDAS_SERVER_HOST - see running on multiple machines. The full list of environment variables midas uses is in Environment variables.

Creating the ODB

Now create the ODB, which holds all the settings and status of the experiment:

odbinit

odbinit checks your environment and exptab, then creates a new, empty ODB in the experiment directory. It is stored in hidden files such as .ODB.SHM. If an ODB already exists, odbinit leaves it alone unless you add the --cleanup option.

Check that everything works by running odbedit, the command-line tool for viewing and editing the ODB. Type ls to list the top level of the ODB, which contains directories such as Experiment, System and Runinfo, then exit or quit to quit:

odbedit
[local:test_expt:S]/>ls
Experiment
System
Programs
Logger
Runinfo
Alarms
Elog
[local:test_expt:S]/>quit

If odbedit can't find your experiment, check the environment variables and the contents of your exptab file.

Run programs

Starting the web server

The web interface is provided by the mhttpd program. Start it in the background with:

mhttpd -D

By default, mhttpd only accepts connections from the same machine, on port 8080 (or 8081 if you used the install script). Point a web browser on the experiment host to http://localhost:8080, and you should see the midas status page.

Connecting from another computer

The default settings deliberately block connections from other computers, as anyone who can reach mhttpd can control your experiment. See web security before making the web interface available on your network.

Starting the logger

The mlogger program writes data files, history data and run log. Start it in the background with:

mlogger -D

The first time it runs, mlogger creates its settings in the /Logger ODB directory. The defaults write each run to a compressed midas file called runNNNNN.mid.lz4 in the experiment directory.

Starting an example frontend

A frontend is a program that reads data from your hardware and sends it to midas. Midas comes with a test frontend called fetest that needs no hardware, and produces some simulated data instead. Start it with:

fetest -D

The status page will now show 5 new equipment entries:

Equipment Data it sends How often
test_random Random data 1Hz, only during a run
test_slow Example slow control data, useful for testing history plots 1Hz, even when a run isn't in progress
test_rare Data that looks like a sine wave 0.1Hz, even when a run isn't in progress
test_bulk Large events 1Hz, only during a run
test_rpc No data. It shows an example of handling remote procedure calls Never

The event counters for test_slow and test_rare start incrementing straight away.

Take data

Taking a first run

Click the Start button on the status page to start a run. The event counters for test_random and test_bulk start incrementing too, and the status page shows the data file growing.

While the run is in progress, you can look at the events being produced in either of two ways:

  • the Event Dump page in the web interface, which shows live events, or
  • the mdump command-line tool, which prints events in the terminal.

Click Stop to end the run. You will find the data file (run00001.mid.lz4 for the first run) in the experiment directory.

Making a first history plot

The history system records the values of equipment variables over time, whether or not a run is in progress.

  1. Go to the History page in the web interface, and click New to create a new plot.
  2. Choose a group and panel name (e.g. Test and Slow).
  3. Add the variables from the test_slow event, and save.

You should see a sine wave, which keeps updating as long as fetest is running. If test_slow isn't listed, check that mlogger is running. See defining plot groups and panels for how to create your own plots, and Web display history panels for all the options.

Automate startup

Writing a start script

Rather than starting each program by hand, write a script that starts everything. For example, save this as /home/daq_user/online/start_daq.sh and make it executable with chmod +x start_daq.sh:

#!/bin/bash
# Start all the midas programs for this experiment

# Set up the environment, as scripts started at boot don't read .bashrc.
# If you used "odbinit --env", you can instead use:
#   source /home/daq_user/online/env.sh
export MIDASSYS=/home/daq_user/packages/midas
export PATH=$PATH:$MIDASSYS/bin
export MIDAS_EXPTAB=/home/daq_user/online/exptab   # or MIDAS_DIR and MIDAS_EXPT_NAME, without an exptab

cd /home/daq_user/online
odbedit -c cleanup   # remove any clients that crashed without disconnecting
mhttpd -D
mlogger -D
fetest -D            # replace with your own frontend(s)

To stop everything, either use the Programs page, or run:

odbedit -c "shutdown all"

Restarting programs from the web interface

You can also tell midas how to start each program, so that it can be restarted from the Programs page, or restarted automatically if it crashes. See Programs: configuring.

Starting midas at boot

The simplest way to start your experiment when the machine boots is to call the start script from your crontab. Run crontab -e and add:

@reboot /home/daq_user/online/start_daq.sh

Note

Only start the programs once. If you use @reboot, don't also start the same programs from a systemd service (the install script sets up a systemd service for mhttpd).

Next steps

Now that the experiment is running, you will probably want to: