# Reviewer 5mdn

Thank you for your response! We are glad that you enjoyed reading our response and about the additional experiments we conducted.

> The XOR experiments are quite nice (i hope they make it into the appendix at the least). Generalization to XOR from OR, AND, and NOT components is not trivial and in my opinion, adds to the experiment.

We will indeed include the XOR results in the next revision.

> Do you find that the appropriate $\tau$ varies by dataset? Some heuristical guidance on how to choose this $\tau$ would be helpful.

We find that an appropriate $\tau$ can vary with the dataset, but only to a lesser extent. For instance, in datasets with less background clutter (e.g. MNIST), we find that smaller $\tau$ can also consistently lead to good in-distribution (validation) performance (cf. Figure 9 in Appendix C.3).

For low compute budget hyper-parameter sweeps, we recommend sweeping $\tau$ between 1.4 and 1.6. In our experiments, we find that values of $\tau$ in this range tend to perform well for all datasets we experiment on. For larger compute budgets, we recommend sweeping between 1.2 and 1.7 (as recommended in line 634 of Appendix A.1.3).

> Can you apply similar disentanglements to NI (via function embeddings)?

For PGM tasks, it is indeed possible to feed the NI with disentangled features of input images (cf. lines 368 and 369). The resulting model (VAE-NI) can then be fairly compared with VAE-WReN, and we anticipate some improvements analogous to those obtained by VAE-WReN over WReN. However, we defer exploring this direction to future work because it has to do with the feature extraction part of the pipeline, which is entirely complementary to the main strength of NIs that we wish to demonstrate with the PGM experiments, namely, compositional reasoning.

> Overall, I quite like this paper and find its contribution significant.

We are glad that you do, and we are grateful for your review! Your comments are proving quite valuable as we build the next revision, and they have encouraged us to explore directions that we would have otherwise missed.

# General Reminders

## Reviewer 8zbe

Dear Reviewer 8zbe,

Thank you again for your review! We believe to have addressed your concerns in our rebuttal comment. Should that not be the case, please do not hesitate to leave us a comment below!

## Reviewer hiUm
Dear Reviewer hiUm,

we thank you again for the time you have invested in reviewing our work! Based on your comments, we have conducted several additional experiments that demonstrate:
1. It is important how parameters are reused, and naïvely reducing the width or depth of the model does not yield the same fast-adaptation performance.
2. The compatibility matrix alone cannot encode enough information to solve adaptation tasks.
3. Pretrained modules are indeed reused at adaptation time.
4. There are non-trivial patterns in how information is routed through the network.

If you have additional questions or if you see issues that we are yet to adequately address in our rebuttal, please do not hesitate to get in touch with us!

## Reviewer zsVd
Dear Reviewer zsVd,

Thank you once again for your review! In our rebuttal comment, we present several clarifications and additional experiments directed towards your questions and concerns. To summarize the latter:

1. We present a visualization of how the modularization learned by Neural Interpreters differs from the one at initialization.
2. We show that adding more layers to ViT does not match the performance improvements obtained by adding more modules to NI.
3. We run an additional batch of experiments to have standard errors on the accuracies accquired on the PGM tasks.

We believe to have addressed your concerns in the rebuttal. Should that not be the case or if you have additional questions, please feel invited to engage with us!