Can multiple components edit parts of the same output in OpenMDAO?

Question

We have a large-ish problem in OpenMDAO where the set of variables exchanged via OpenMDAO would be much smaller if multiple different components could write to parts of the same variable name(s). Specifically:

1. 4 different components are contributing to the overall cost values, to simplify this we could have a single cost variable that has length=4 and then each of these components would write to the corresponding index within the same cost variable.
2. For another part of the code, there are multiple near-duplicate components that ideally populate different elements of a given vector with a physical parameter (voltage). In this case we would like to scale the vector length with problem size. This size would be known at run time, but it would be helpful to not need to have a huge list of variables that changes with this scale.
   • To complete the picture, an earlier-to run component creates the vector used as a common input for these, and they can easily index them.
   • And downstream, another component aggregates the results, such that resorting to unique names per near-duplicate component might otherwise mean having to subscribe to a long and variable length set of variables names.

Ideally, for both cases if components could stably each write different aspects of the same variable and have other components that use this variable as an input simply get the most recently updated set of data. If this is not possible a work around for the fixed cost example (#1) could be to just name all of the pieces, but for the variable-length part (#2) seems we would need to create some form of wrapper around the sub-problems so it would look like a single component. But neither are quite as clean.

I have seen hints at this type of capability in the MPI support, but we are currently not planning to use mpi here for simplicity and for cross-platform ease (e.g. seems MPI can be challenging under windows, we also need run under OSX and Linux?)

Is this possible? Suggestions?

Answer 1

As of OpenMDAO V3.26 the functionality you are asking for does not exist. In OpenMDAO you can have src_indices which specify a sub-set of the source array to map into a component input. However, you can only link a single src (output) to a single target (input). Here is how they work with the help of a hastily drawn ascii art diagram:

The src array (output) has a length of 7. The target array (input) has a length of 4. So this connection would have a src_indices argument of [0,3,4,6]. During a data transfer OpenMDAO will copy those indices into a continuous array on the tgt side. The key here is that there is only one connection, and whatever the values of the src_indices are on the source side of the connection you get only a single continuous array on the target side.

As of V3.26 This is all that OpenMDAO allows. It is theoretically possible to modify OpenMDAO to support a more complex connection scheme, but the devil is in the details and as of May 2023 there is no current POEM that proposes it. This feature has been discussed before though, particularly at the 2022 OpenMDAO workshop in Cleveland OH. So if you are interested in the feature I recommend you consider authoring a POEM. To help you, I'll quickly address a few key details you'd need to consider.

What you are asking for is to allow multiple sources (outputs) to connect to a single target (inputs). Even if OpenMDAO allowed this situation, the existing src_indices data is not enough information to complete the connection. You would need to consider where that data is to go in the target array as well. There are two options:

1. force users to ALSO tell you where it should go via an additional tgt_indices argument. This would let you specify any arbitrary mapping of source to target. However, it would potentially make error checking very tricky and leaves open the question of what to do if you somehow get unassigned indices in the target array. What if two sources accidentally specify the same target index and collide?

2. keep all data pushed into the target array as continuous with respect to that connection. In this case, no target indices would be needed in general. However, you'd have a problem knowing which order the data was in target array then. So you'd either have to add some kind of ordering information to the connection via a seq_index argument in the connection (which would basically be 1 for all normal connections that didn't do any multi-source behavior), have the order be defined implicitly by the sequence the connections were defined int, or come up with some other magic scheme that I haven't thought of :)

Either way --- (1) or (2) --- you might also want to consider if the inside the compute of the target component, you'd need to have access to the sequence information at all. Components don't currently have any kind of source information inside their compute, but I imagine that it might be needed in this case.

I bring all this up to emphasize that, despite being technically possible, the issue isn't as trivial as it might seem. I have also run into cases where it would have been useful myself though. So I see the issue from both sides.

What do I suggest you do?

Having given that rather long winded explaination above, my actual advice keys off these two points that you raised:

To complete the picture, an earlier-to run component creates the vector used as a common input for these, and they can easily index them.

And downstream, another component aggregates the results, such that resorting to unique names per near-duplicate component might otherwise mean having to subscribe to a long and variable length set of variables names.

Given that you have a vectorized component at the start of the calculation, and another one at the end, I can't easily understand why the middle calculation itself needs to be a series of separate components. Could it not be a vectorized component itself?

For the cost calculation, why not move all four calculations into a single component? It should be mostly a issue of consolidating the 4 different compute methods into a single one. Especially of these 4 components don't share any connections with eachother then this should be completely trivial, even accounting for any analytic derivatives you've implemented.

For the voltage component, again I ask why not make it a single vectorized component that handles the sets of nearly identicle calculations inside a single compute?

Less Components are Generally Better

Even if some kind of tgt_indices functionality existed, I would still give the same advice. Generally speaking OpenMDAO has some for loops that (very roughly speaking) are nested by first looping over components then over the variables. So as a rule of thumb you're better off from a framework overhead prespective with fewer vectorized components and less overall variables.