Hi! I want to get the validation and test accuracy as Table 4 in "NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size" paper. I just want to check the following commands are correct or not:

After finishing the architecture search (I'm studying Weight-Sharing approach), I get the genotype. Then, I get the arch_index via:
arch_index = api.query_index_by_arch('......genotype here......')
Therefore, for Cifar10 validation accuracy:
info = api.get_more_info(arch_index, 'cifar10-valid', hp=200)
for Cifar10 test accuracy:
info = api.get_more_info(arch_index, 'cifar10', hp=200)
for Cifar100 validation/test accuracy:
info = api.get_more_info(arch_index, 'cifar100', hp=200)
for ImageNet16-120 validation/test accuracy:
info = api.get_more_info(arch_index, 'ImageNet16-120', hp=200)

Following are some points I want to check:

1. Does Cifar10 test accuracy results use train + valid set for training and test set for testing? Thus, I should use 'cifar10' instead of 'cifar10-valid' to get test accuracy?
2. What does valtest-accuracy mean in Cifar100 and ImageNet16-120?
3. I get the architecture with validation accuracy higher than Optimal values reported in the paper for ImageNet16-120. Why?
   

Great Thanks!