Odbxx: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
A C++11 object-oriented interface to the [[ODB|ODB (online database)]] was introduced in May 2020. You can find more details about the ODB on the [[ODB_Access_and_Use|ODB Access and Use]]  page, which includes links to the command-line, javascript, python, and non-object C++ interfaces.
A C++11 object-oriented interface to the [[ODB|ODB (online database)]] was introduced in May 2020. You can think of it like a "magic" map/dictionary that automatically sends changes you make to the ODB, and receives updates that others have made.


The header for this odbxx interface is at [https://bitbucket.org/tmidas/midas/src/develop/include/odbxx.h odbxx.h] and example usage in [https://bitbucket.org/tmidas/midas/src/develop/progs/odbxx_test.cxx odbxx_test.cxx]
The header for this odbxx interface is at [https://bitbucket.org/tmidas/midas/src/develop/include/odbxx.h odbxx.h] and example usage in [https://bitbucket.org/tmidas/midas/src/develop/progs/odbxx_test.cxx odbxx_test.cxx].
 
You can find more details about the ODB on the [[ODB_Access_and_Use|ODB Access and Use]]  page, which includes links to the command-line, javascript, python, and non-object C++ interfaces.


== Basic usage ==
== Basic usage ==
Line 7: Line 9:
The simplest usage is like:
The simplest usage is like:


  // Grab a bit of the ODB
<pre>// Grab a bit of the ODB
  midas::odb exp("/Experiment");
midas::odb exp("/Experiment");
 
 
  // Simple read
// Simple read
  std::cout << "The current transition timeout is " << exp["Transition timeout"] << std::endl;
std::cout << "The current transition timeout is " << exp["Transition timeout"] << std::endl;
 
 
  // Make a change. The new value is automatically sent to the ODB.
// Make a change. The new value is automatically sent to the ODB.
  // Most C++ operators are supported (++, += etc), or you can do a simple
// Most C++ operators are supported (++, += etc), or you can do a simple
  // re-assignment like `exp["Transition timeout"] = 12345;`.
// re-assignment like `exp["Transition timeout"] = 12345;`.
  exp["Transition timeout"] += 100;
exp["Transition timeout"] += 100;
 
 
  // Read the new value
// Read the new value
  std::cout << "The transition timeout is now " << exp["Transition timeout"] << std::endl;
std::cout << "The transition timeout is now " << exp["Transition timeout"] << std::endl;
</pre>


You can automatically cast to regular data types (int, double) etc if you want a copy of the value to work with:
You can automatically cast to regular data types (int, double) etc if you want a copy of the value to work with:


  int curr_timeout = exp["Transition timeout"];
<pre>int curr_timeout = exp["Transition timeout"];</pre>
 


== Automatic refreshing ==
== Automatic refreshing ==


You may temporarily disable the automatic updating to/from the ODB
You may temporarily disable the automatic updating to/from the ODB
using `odb::set_auto_refresh_write(false)` and `odb::set_auto_refresh_read(false)`.
using <code>odb::set_auto_refresh_write(false)</code> and <code>odb::set_auto_refresh_read(false)</code>.


If auto-refresh is enabled (the default), your new values are sent to
If auto-refresh is enabled (the default), your new values are sent to
the ODB as soon as you touch the value in the `midas::odb` object. The ODB
the ODB as soon as you touch the value in the <code>midas::odb</code> object. The ODB
is queried for new values whenever you access the value. In the above
is queried for new values whenever you access the value. In the above
example, the ODB is queried 4 times (during construction of `exp`, and
example, the ODB is queried 4 times (during construction of <code>exp</code>, and
each time `exp["Transition timeout"]` is mentioned), and written to 1
each time <code>exp["Transition timeout"]</code> is mentioned), and written to 1
time (when `exp["Transition timeout"]` is assigned to).
time (when <code>exp["Transition timeout"]</code> is assigned to).


See the "Callback functions" section below for details on how to have a function
See the [[#Callback functions]] section below for details on how to have a function
called when a value changes.
called when a value changes.


== Arrays/vectors ==
== Arrays/vectors ==
Line 48: Line 49:
You can access/edit individual elements using []:
You can access/edit individual elements using []:


  odb["Example"][1] = 1.2;
<pre>odb["Example"][1] = 1.2;</pre>


You can completely re-assign content using a std::vector or std::array:
You can completely re-assign content using a std::vector or std::array:


  std::vector<float> vec = std::vector<float>(10);
<pre>std::vector<float> vec = std::vector<float>(10);
  odb["Example"] = vec;
odb["Example"] = vec;
</pre>


You can resize arrays using `odb::resize()`. If the existing array is longer,
You can resize arrays using <code>odb::resize()</code>. If the existing array is longer,
it will be truncated; if shorter it will be extended with default values
it will be truncated; if shorter it will be extended with default values
(0 or an empty string).
(0 or an empty string).


  odb["Example"].resize(5); // Now is 5 elements long
<pre>odb["Example"].resize(5); // Now is 5 elements long</pre>


Note that arithmetic operators are supported for arrays, and will apply
Note that arithmetic operators are supported for arrays, and will apply
the operation to ALL ELEMENTS IN THE ARRAY:
the operation to ALL ELEMENTS IN THE ARRAY:


  // Create the vector
<pre>// Create the vector
  std::vector<float> vec = std::vector<float>(2);
std::vector<float> vec = std::vector<float>(2);
  vec[0] = 3;
vec[0] = 3;
  vec[1] = 5;
vec[1] = 5;
 
 
  // Assign in ODB
// Assign in ODB
  odb["Example"] = vec;
odb["Example"] = vec;
 
 
  // Multiply ALL elements by 2
// Multiply ALL elements by 2
  odb["Example"] *= 2;
odb["Example"] *= 2;
 
 
  // odb["Example"] now contains {6, 10}.
// odb["Example"] now contains {6, 10}.
</pre>


You can directly iterate over arrays/vectors:
You can directly iterate over arrays/vectors:


  // Iterating using standard begin/end.
<pre>// Iterating using standard begin/end.
  for (auto it = o["Int Array"].begin(); it != o["Int Array"].end(); it++) {
for (auto it = o["Int Array"].begin(); it != o["Int Array"].end(); it++) {
      int val = *it;
  int val = *it;
      std::cout << val << std::endl;
  std::cout << val << std::endl;
  }
}
 
</pre>
  // Iterating using C++11 range-based for loop.
  for (int val : o["Int Array"]) {
      std::cout << val << std::endl;
  }


<pre>// Iterating using C++11 range-based for loop.
for (int val : o["Int Array"]) {
  std::cout << val << std::endl;
}
</pre>


== Strings ==
== Strings ==


Strings in the ODB are returned as std::string (unlike the midas.h db_get_value()
Strings in the ODB are returned as std::string (unlike the midas.h <code>db_get_value()</code>
family of functions, where strings are returned as char*). You may have vectors of strings.
family of functions, where strings are returned as char*). You may have vectors of strings.


== Creating new bits of the ODB ==
== Creating new bits of the ODB ==


You can automatically create bits of the ODB by passing a struct to the
You can automatically create bits of the ODB by passing a struct to the
`midas::odb` constructor, then calling `odb::connect()`, like:
<code>midas::odb</code> constructor, then calling <code>odb::connect()</code>, like:
 
<pre>// Define the ODB structure
midas::odb new_bit = {
  {"Int32 Key", 42},
  {"Bool Key", true},
  {"Subdir", {
      {"Float key", 1.2f},    // floats must be explicitly specified
  }},
  {"Int Array", {1, 2, 3}},
  {"Double Array", {1.2, 2.3, 3.4}},
  {"String Array", {"Hello1", "Hello2", "Hello3"}},
  {"Large Array", std::array<int, 10>{} },  // array with explicit size
  {"Large String", std::string(63, '\0') },  // string with explicit size
};
 
// Then sync the structure. This function
// - keeps the existing value of any keys that are in the ODB and your code
// - creates any keys that are in your code but not yet in the ODB
o.connect("/Test/Settings");
 
// If you make the `write_defaults` argument true, then the function
// - overwrites the value of any keys that are in the ODB with the value in your code
// - creates any keys that are in your code but not yet in the ODB
o.connect("/Test/Settings", true);


  // Define the ODB structure
// The `connect_and_fix_structure()` method acts like the old db_check_record() function, and
  midas::odb new_bit = {
// - keeps the existing value of any keys that are in the ODB and your code
      {"Int32 Key", 42},
// - creates any keys that are in your code but not yet in the ODB
      {"Bool Key", true},
// - deletes any keys that are in the ODB but not your code
      {"Subdir", {
// - updates the order of keys in the ODB to match your code
        {"Float key", 1.2f},    // floats must be explicitly specified
o.connect_and_fix_structure("/Test/Settings");
      }},
</pre>
      {"Int Array", {1, 2, 3}},
      {"Double Array", {1.2, 2.3, 3.4}},
      {"String Array", {"Hello1", "Hello2", "Hello3"}},
      {"Large Array", std::array<int, 10>{} },  // array with explicit size
      {"Large String", std::string(63, '\0') }, // string with explicit size
  };
 
  // Then sync the structure. Any keys that don't exist will be created; any
  // that already exist will keep their existing values...
  o.connect("/Test/Settings");
 
  // ... unless you make the `write_defaults` argument true, in which case the
  // existing values will be ignored, and overwritten with what you specified above.
  o.connect("/Test/Settings", true);


If you want to add new keys to existing ODB subdirectories, you can also just use the [] operator:
If you want to add new keys to existing ODB subdirectories, you can also just use the [] operator:


  midas::odb existing_key("/MyExistingKey");
<pre>midas::odb existing_key("/MyExistingKey");
  existing_key["MyNewSubKey"] = 1.23;
existing_key["MyNewSubKey"] = 1.23;
</pre>


You can also create new keys by providing a default value when reading a value. If the key doesn't already exist, the default value will be used.
You can also create new keys by providing a default value when reading a value. If the key doesn't already exist, the default value will be used.


  midas::odb existing_key("/MyExistingKey");
<pre>midas::odb existing_key("/MyExistingKey");
  double val = existing_key["MyNewSubKey"](1.23);
double val = existing_key["MyNewSubKey"](1.23);
</pre>


== Iterating over subkeys ==
== Iterating over subkeys ==


You can use iterate over subkeys using normal iterator functions.
You can iterate over subkeys using normal iterator functions.


  // Iterating using standard begin/end.
<pre>// Iterating using standard begin/end.
  midas::odb exp("/Experiment");
midas::odb exp("/Experiment");
 
  for (auto it = exp.begin(); it != exp.end(); it++) {
      midas::odb& subkey = *it;
      std::cout << subkey.get_name() << " = " << subkey << std::endl;
  }


  // Iterating using C++11 range-based for loop.
for (auto it = exp.begin(); it != exp.end(); it++) {
   for (midas::odb& subkey : exp) {
   midas::odb& subkey = *it;
      std::cout << subkey.get_name() << " = " << subkey << std::endl;
  std::cout << subkey.get_name() << " = " << subkey << std::endl;
  }
}
</pre>


You can check whether a subkey exists using `odb::is_subkey()`.
<pre>// Iterating using C++11 range-based for loop.
for (midas::odb& subkey : exp) {
  std::cout << subkey.get_name() << " = " << subkey << std::endl;
}
</pre>


You can check whether a subkey exists using <code>odb::is_subkey()</code>.


== Deleting bits of the ODB ==
== Deleting bits of the ODB ==


You can use `odb::delete_key()` to remove part of the ODB:
You can use <code>odb::delete_key()</code> to remove part of the ODB:
 
  midas::odb existing_bit("/Some/ODB/Path");
  existing_bit.delete_key();


<pre>midas::odb existing_bit("/Some/ODB/Path");
existing_bit.delete_key();
</pre>


== Callback functions ==
== Callback functions ==


You may also set up callback functions that are called whenever a value
You may also set up callback functions that are called whenever a value
changes, using the `odb::watch()` function. Note that you must call
changes, using the <code>odb::watch()</code> function. Note that you must call
`cm_yield()` (from midas.h) periodically for this to work - deep down it
<code>cm_yield()</code> (from midas.h) periodically for this to work - deep down it
is `cm_yield()` itself that calls your callback function.
is <code>cm_yield()</code> itself that calls your callback function.


The callback functions can either be a "normal" function or a C++ lambda.
The callback functions can either be a "normal" function or a C++ lambda.
In either case, it should accept one argument - a `midas::odb` object (passed
In either case, it should accept one argument - a <code>midas::odb</code> object (passed
by reference) that contains the new state.
by reference) that contains the new state.


  // Example with a lambda:
<pre>// Example with a lambda:
  midas::odb to_watch("/Experiment");
midas::odb to_watch("/Experiment");
  to_watch.watch([](midas::odb &arg) {
to_watch.watch([](midas::odb &arg) {
      std::cout << "Value of key \"" + arg.get_full_path() + "\" changed to " << arg << std::endl;
  std::cout << "Value of key \"" + arg.get_full_path() + "\" changed to " << arg << std::endl;
  });
});
 
</pre>
  // Example with a "normal" function:
  void my_function(midas::odb &arg) {
      std::cout << "Value of key \"" + arg.get_full_path() + "\" changed to " << arg << std::endl;
  }
    
    
  midas::odb to_watch("/Experiment");
<pre>// Example with a "normal" function:
   to_watch.watch(my_function);
void my_function(midas::odb &arg) {
   std::cout << "Value of key \"" + arg.get_full_path() + "\" changed to " << arg << std::endl;
}


midas::odb to_watch("/Experiment");
to_watch.watch(my_function);
</pre>


== Example code ==
== Example code ==


A full working example exploring most of the features can be found in
A full working example exploring most of the features can be found in
`progs/odbxx_text.cxx`. The test executable will be compiled as
<code>odbxx/odbxx_test.cxx</code>. The test executable will be compiled as
`build/progs/odbxx_test` (it is not installed in the `bin` directory).
<code>build/odbxx/odbxx_test</code> (it is not installed in the `bin` directory).

Revision as of 10:22, 6 May 2021

A C++11 object-oriented interface to the ODB (online database) was introduced in May 2020. You can think of it like a "magic" map/dictionary that automatically sends changes you make to the ODB, and receives updates that others have made.

The header for this odbxx interface is at odbxx.h and example usage in odbxx_test.cxx.

You can find more details about the ODB on the ODB Access and Use page, which includes links to the command-line, javascript, python, and non-object C++ interfaces.

Basic usage

The simplest usage is like:

// Grab a bit of the ODB
midas::odb exp("/Experiment");

// Simple read
std::cout << "The current transition timeout is " << exp["Transition timeout"] << std::endl;

// Make a change. The new value is automatically sent to the ODB.
// Most C++ operators are supported (++, += etc), or you can do a simple
// re-assignment like `exp["Transition timeout"] = 12345;`.
exp["Transition timeout"] += 100;

// Read the new value
std::cout << "The transition timeout is now " << exp["Transition timeout"] << std::endl;

You can automatically cast to regular data types (int, double) etc if you want a copy of the value to work with:

int curr_timeout = exp["Transition timeout"];

Automatic refreshing

You may temporarily disable the automatic updating to/from the ODB using odb::set_auto_refresh_write(false) and odb::set_auto_refresh_read(false).

If auto-refresh is enabled (the default), your new values are sent to the ODB as soon as you touch the value in the midas::odb object. The ODB is queried for new values whenever you access the value. In the above example, the ODB is queried 4 times (during construction of exp, and each time exp["Transition timeout"] is mentioned), and written to 1 time (when exp["Transition timeout"] is assigned to).

See the #Callback functions section below for details on how to have a function called when a value changes.

Arrays/vectors

ODB arrays are represented by std vectors.

You can access/edit individual elements using []:

odb["Example"][1] = 1.2;

You can completely re-assign content using a std::vector or std::array:

std::vector<float> vec = std::vector<float>(10);
odb["Example"] = vec;

You can resize arrays using odb::resize(). If the existing array is longer, it will be truncated; if shorter it will be extended with default values (0 or an empty string).

odb["Example"].resize(5); // Now is 5 elements long

Note that arithmetic operators are supported for arrays, and will apply the operation to ALL ELEMENTS IN THE ARRAY:

// Create the vector
std::vector<float> vec = std::vector<float>(2);
vec[0] = 3;
vec[1] = 5;

// Assign in ODB
odb["Example"] = vec;

// Multiply ALL elements by 2
odb["Example"] *= 2;

// odb["Example"] now contains {6, 10}.

You can directly iterate over arrays/vectors:

// Iterating using standard begin/end.
for (auto it = o["Int Array"].begin(); it != o["Int Array"].end(); it++) {
   int val = *it;
   std::cout << val << std::endl;
}
// Iterating using C++11 range-based for loop.
for (int val : o["Int Array"]) {
   std::cout << val << std::endl;
}

Strings

Strings in the ODB are returned as std::string (unlike the midas.h db_get_value() family of functions, where strings are returned as char*). You may have vectors of strings.

Creating new bits of the ODB

You can automatically create bits of the ODB by passing a struct to the midas::odb constructor, then calling odb::connect(), like:

// Define the ODB structure
midas::odb new_bit = {
   {"Int32 Key", 42},
   {"Bool Key", true},
   {"Subdir", {
      {"Float key", 1.2f},     // floats must be explicitly specified
   }},
   {"Int Array", {1, 2, 3}},
   {"Double Array", {1.2, 2.3, 3.4}},
   {"String Array", {"Hello1", "Hello2", "Hello3"}},
   {"Large Array", std::array<int, 10>{} },   // array with explicit size
   {"Large String", std::string(63, '\0') },  // string with explicit size
};

// Then sync the structure. This function
// - keeps the existing value of any keys that are in the ODB and your code
// - creates any keys that are in your code but not yet in the ODB
o.connect("/Test/Settings");

// If you make the `write_defaults` argument true, then the function
// - overwrites the value of any keys that are in the ODB with the value in your code
// - creates any keys that are in your code but not yet in the ODB
o.connect("/Test/Settings", true);

// The `connect_and_fix_structure()` method acts like the old db_check_record() function, and
// - keeps the existing value of any keys that are in the ODB and your code
// - creates any keys that are in your code but not yet in the ODB
// - deletes any keys that are in the ODB but not your code
// - updates the order of keys in the ODB to match your code
o.connect_and_fix_structure("/Test/Settings");

If you want to add new keys to existing ODB subdirectories, you can also just use the [] operator:

midas::odb existing_key("/MyExistingKey");
existing_key["MyNewSubKey"] = 1.23;

You can also create new keys by providing a default value when reading a value. If the key doesn't already exist, the default value will be used.

midas::odb existing_key("/MyExistingKey");
double val = existing_key["MyNewSubKey"](1.23);

Iterating over subkeys

You can iterate over subkeys using normal iterator functions.

// Iterating using standard begin/end.
midas::odb exp("/Experiment");

for (auto it = exp.begin(); it != exp.end(); it++) {
   midas::odb& subkey = *it;
   std::cout << subkey.get_name() << " = " << subkey << std::endl;
}
// Iterating using C++11 range-based for loop.
for (midas::odb& subkey : exp) {
   std::cout << subkey.get_name() << " = " << subkey << std::endl;
}

You can check whether a subkey exists using odb::is_subkey().

Deleting bits of the ODB

You can use odb::delete_key() to remove part of the ODB:

midas::odb existing_bit("/Some/ODB/Path");
existing_bit.delete_key();

Callback functions

You may also set up callback functions that are called whenever a value changes, using the odb::watch() function. Note that you must call cm_yield() (from midas.h) periodically for this to work - deep down it is cm_yield() itself that calls your callback function.

The callback functions can either be a "normal" function or a C++ lambda. In either case, it should accept one argument - a midas::odb object (passed by reference) that contains the new state.

// Example with a lambda:
midas::odb to_watch("/Experiment");
to_watch.watch([](midas::odb &arg) {
   std::cout << "Value of key \"" + arg.get_full_path() + "\" changed to " << arg << std::endl;
});
// Example with a "normal" function:
void my_function(midas::odb &arg) {
   std::cout << "Value of key \"" + arg.get_full_path() + "\" changed to " << arg << std::endl;
}

midas::odb to_watch("/Experiment");
to_watch.watch(my_function);

Example code

A full working example exploring most of the features can be found in odbxx/odbxx_test.cxx. The test executable will be compiled as build/odbxx/odbxx_test (it is not installed in the `bin` directory).