We're really grateful to the reviewers for their careful reviews and helpful feedback.

We begin by addressing some concerns raised by multiple reviewers, and then respond to each reviewer in turn. The changes made in the revision highlight the indices of specific concerns below.

## Common concerns

> C1. Core contributions

MASE contains three research contributions:

* A novel IR (only software part extended from Torch FX) that describes *both* software and hardware at module level;
* Scalable hardware synthesis with cross-device design space exploration that maps a model onto a system with any number of hardware acceleartors; and
* Two case studies to demonstrate how to use MASE for auto quantization of large ML models at scale.

We have clarified that in Section 1 and added a comparison with other IRs in Section 3.2.


> C2. Background

We have more related works about ML software and hardware co-design.

> C3. Operation granularity

Thanks for the comments and suggestions. The operations/nodes in MASE IR are at module level. We have added detailed language specificaitions of MASE IR to Table 1 and improved the example in Figure 2.

> C4. Design Space Exploration

Thanks for the comments and suggestions. We have rewritten Section 4.3.3 by adding the following explanations:
* Deatiled explanation about the internal functions shown in Algorithm 1;
* Deatiled explanation about the software and hardware parameters for searching;
* Explanation of the device-level partitioning and mapping, and examples of breakdown in the DSE plot;
* Explaination of how to find Pareto front designs (JC: Maybe just say the pfd is manually determined);
* How synthesis is carried out during the DSE and how it is optimised; and
* Explaination the effects of portioning and how are they dealt in the DSE process.

> C5. Figure 9 and Figure 10

We have improved the figures with a more intuitive precision distribution with more explanation to Section 5.

> Syntactical and grammatical errors.

Fixed.

## Reviewer A

> Knowledge distillation (KD)

Thanks for pointing us to this reference. We agree that techniques such as KD can further improve the training accuracy but KD is out of the scop of this work. To ensure fairness, both baselines and our results were not optimised using any of these tricks.We have clarify that in Section 5.

> Figure 8: It is reported to depict a comparison between MASE-produced hardware and optimized designs obtained from internal blocks. However, the figure does not effectively convey this comparison, as there is only one bar per metric and ML model, without a second bar for different hardware.

JC: Plan to ignore this...

## Reviewer C

> Quantization looks trivial.

The quantization is a case study of how MASE can be useful and it has already shown promising results compared to GPUs. One of our key research contributions is that MASE *enables* such software and hardware co-optimisation at scale, using simple commands as shown in Figure 1. We have included the commands for the case study to show how easy MASE is to use.

<span style="color:red">*[AZ: Maybe also something on the line of "Quantisation may have been a research topic that had been previously explored, but it comes to present a huge challenge in today's DSE engines. If we consider X (fill number) quantisation choices for each layer in a Y-layer network, we have X^Y search space; coupled with the already-existing hardware design space of size XXX, this results in an incredibly vast search space of size XXX. This case study demonstrated the scalability of our DSE, illustrating how quantisation – a highly modular and orthogonal search space – can be seamlessly integrated."]*</span>

## Reviewer D

> The HLS results are bad.

Thanks for the comments. We agree HLS today can produce hardware with comparable hardware by combining existing HLS *optimisation* tools such as ScaleHLS or Polsca. The HLS end in MASE is only to show the capability of synthesising arbitrary ML models. The emitted MLIR is directly translated to C code and sent HLS *synthesis* tool, Vivado HLS. The quality of design is not comparable to the internal design. Interactions with HLS optimisation tools is a seperate problem and could be an interesting future work. We have clarified that in the revised manuscript.

> Figure 7.

Reduced.

> Section 3.1.

Removed.

> MLIR Lowering

MASE lowers PyTorch model to Linalg and then affine. The affine MLIR is then emitted into HLS code in C.
We have clarified that in the revised manuscript.

> Co-simulation for system level integration.

Sorry about the misleading sentence. MASE supports the integration of these components but for fast testing, MASE also provides a user option to hide them. We have clarify that in the revised revison.

> NITs:
>
> * Generally, questions seem to be phrased as "how to...?" which usually doesn't read as a question.
> * 4.3.1, para1. "well-optimised to" and "efficiency with" both are grammatically incorrect

Fixed.

## Reviewer E

> Many accelerators system

Existing works for mapping a model onto multi-FPGAs are limited, so we compare our approach with single-FPGA and GPU results. Our results are presented as multi-FPGA results and we have added detailed explanations on the board the count and how were they obtained.

> Performance and area model in DSE

The performance model is pre-defined for internal operations and MASE assumes the input and output throughput of an unknown op are the same by default but can also be specified by the user. The area model is using linear regression for the internal op but requires synthesis during DSE for unknown ops unless provided by the user. In our experiments, we observe that MASE already support most of the operations in each model which significantly reduce the search time <span style="color:red">*[AZ: "reduce the search time by N times, and this is demonsrated in Section X in the paper": I wouold point out where it is, and try to give exact numbers.]*</span>
. We have further clarified that in the revised manuscript.

> MLIR cannot be trained or tuned?

Yes. Both software and hardware optimisations are performed in MASE IR so they can be interleaved. We have clarified that in Section 4.1. MLIR is an exit point of the MASE flow.