Currently we use the following for classification:

• We sample 8 training examples per class, compute the embeddings, and fit the LogReg classifier. We then evaluate on the (unchanged) dev / test set.
• We repeat the previous step 10 times and compute the average for accuracy / f1 etc.

As the test set embeddings will be the same, we can compute the test set embeddings once and just need to feed them to the LR classifier. This will make the 10-times repeated evaluation much faster.