What do I need active subspaces for?
Suppose you have an expensive computer simulation with lots of parameters and you need to do some sort of parametric study—like optimization or uncertainty quantification. Methods for such studies do not fare well in more than a handful of dimensions. Any method that claims to operate in high dimensions must exploit some low-dimensional structure in the problem: sparse grids exploit a particular kind of coordinate-oriented smoothness, low-rank methods exploit separable structure, adaptive methods exploit a model’s dependence on a few of its many parameters. Active subspaces identify a particular kind of exploitable low-dimensional structure. If your model admits such structure, then otherwise infeasible parametric studies become possible.
How is active subspaces different from principal component analysis?
Principal component analysis seeks a low-dimensional linear parameterization of a random vector. In contrast, active subspaces help approximate a scalar-valued function of several variables. To construct the subspace, we use the eigenvectors of something that looks like the uncentered covariance of the gradient. But we don’t care about constructing a low-dimensional, linear parameterization of the gradient such that its covariance is well-approximated—like in PCA. Instead, we use those eigenvectors to identify directions along which the scalar-valued function changes more, on average.
How many samples of the gradient do I need to estimate the active subspace?
In our paper Computing active subspaces, we analyze a Monte Carlo method for estimating the eigenvectors. We use nonasymptotic random matrix theory—specifically, some matrix concentration inequalities—developed by Alex Gittens and Joel Tropp to show that the number of samples needed to estimate the kth eigenvalue scales like the log of the dimension. In practice, we use between 2 and 10 times k times the log of the dimension.
What if I don’t have access to gradients?
We define the active subspace with the gradient. If you don’t have the gradient, then you need to approximate it—like with finite differences. Another option for noisy simulations is to use local, least-squares-fit linear models constructed with samples near the location of the desired gradient evaluation. In practice, we often gain insight using the gradient of a global, least-squares-fit linear model. In the statistics literature, there are several methods for estimating the related central subspace: sliced inverse regression, principal Hessian directions, minimum average variance estimation, etc. The problem set up and interpretation is a bit different in this case, but the methods show promise for identifying the active subspace without gradients.
What sort of simulation models admit an active subspace?
It’s hard to say without testing. The idea behind active subspaces is to test a given model using its gradients. We’ve found active subspaces in models from aerospace, climate, hydrology, and renewable energy. However, it’s very easy to construct functions that do not admit an active subspace. A quadratic function whose Hessian is the identity matrix doesn’t have an active subspace.
Active subspaces are derived from a scalar-valued function. Can they extend to vector-valued functions, i.e., multiple quantities of interest?
If you have two or three quantities of interest, look for active subspaces in each one separately and compare the results. We have compared active subspaces from lift and drag in aerospace vehicles, and this comparison often provides insights into the model. If your simulation output is truly vector-valued, it’s easy enough to extend the construction by swapping the vector-valued gradient with the matrix-valued Jacobian. However, it’s not clear how to interpret or exploit the resulting subspaces. Also, when we’ve tried this, the dimension of the active subspace tends to be a lot larger—so the dimension reduction is not as dramatic.
If you have a question you’d like answered in the FAQ, send an email to Paul, Qiqi, Youssef, or Tan. You can find our emails at our respective websites on the PEOPLE page.