We're really grateful to the reviewers for their careful reviews and helpful feedback. We've linked the changes in the revised manuscript to the indices of the questions below. We would appreciate it if the reviewers could read our replies with the updated manuscript opened on the side.

### Reviewer-A

Thanks for the detailed comments.

> RA.1 Section-4.1

We've added more details about IR and parameters accordingly in Figure-2(IR and its actions), Table-2(parameters) and Section-4.1.

> RA.2 Section-4.2

MASE IR is the only *trainable hardware IR* for ML. We've added Table-1 and a comparison with other IRs in Section-3.2.

> RA.3 Section-4.3.1

We agree that the toolflow presented in Section-4.3.1 looks technical, but this enables our research contribution on DSE for *multi-accelerator systems*.

> RA.4 Section-4.3.3(now 4.3.2)

We've formalised the DSE problem and clarified the original internal functions in Section-4.3.2.

> RA.5 Section-5

As KD was not used in this work, we agree that referring to it is unnecessary. We've removed KD to avoid confusion.

> RA.6 Figure-7(now Figure-5)

The Pareto front points are defined as hardware where all the throughput of nodes is balanced.
We manually determined them based on this definition, and we've clarified that in the caption.

> RA.7 Figure-8(removed)

It has been removed to avoid confusion.

> RA.8 Figure-9(now Figure-8) & Figure-10(now Figure-9)

Sorry for the mistake. Figure-8 shows the hardware design quality from the software quantisation search, and Figure-9 shows the layer breakdown of precisions of the best hardware design. We've improved Figure-9 and updated the captions for both figures to avoid confusion.

### Reviewer-B

Thanks for your positive comments.

### Reviewer-C

> RC.1 Parameter

We've added more details about parameters accordingly in Table-2 and improved examples in Figure-2.

> RC.2 Quantisation

This work focuses on the MASE framework(IR and DSE for multi-accelerator systems), and we use quantisation just for an example application. Still, MASE has shown promising results just with quantisation.

> RC.3 related works

Thanks for the comments. We've added more co-design-related works to Section-3.3.

> RC.4 Core contribution

Our contributions are both. We've improved our results to show these contributions significantly improve *the scalability* of hardware-accelerator system synthesis in Figure 5.

>RC.5 Multiple devices

We've clarified the device partitioning during DSE in Section-4.3.2 and highlighted the device count in the final results(Table-6 & Table-7).

### Reviewer-D

> RD.1 IR and parameters

MASE IR is the only *trainable hardware IR* for ML. We've implemented more than 50 attributes and 20 passes, and presented the key ones in improved Table 2.

> RD.2 HLS underpeform

MASE aims to support the hardware synthesis of most models on HuggingFace. These models are updated quickly with irregular ops not covered in our standard ops. We use HLS for these ops to enable synthesisablity for arbitrary or new ML models.

> RD.3 Figure-7(now Figure-5)

We've improved Figure-5.

>RD.4 Section-3.1

Deleted old Section-3.1.

> RD.5 Section-4.3.1

Yes. We use existing passes to lower linalg to affine, now clarified in Section-4.3.1.

> RD.6 Testing

Sorry about the misleading sentence. MASE supports simulation with these components and also offers a user option for fast behaviour testing. We've clarified that in the revised version.

> NITS

Thanks for your detailed review. All fixed.

### Reviewer-E

Thanks for the detailed comments.

> RE.1 1) Many accelerators system

Sorry about the confusion. The results are for multiple FPGAs. We've highlighted the device count in Figure-5, Table-6 and Table-7.

> RE.2 2) Parameter list

We've improved the parameter list in Table-2 and added an example to Figure-2. We use a regression model for pre-defined components for improving scalability and clarified that in Section 4.3.2. The accuracy of our regression model is provided in the newly added Table-4.

> RE.3 3) Figure-7(now Figure-5)

The dots are the trace of the DSE process. The Pareto front points(blue dots) are defined as hardware where all the throughput of nodes are balanced. We manually determined them based on this definition, and we've clarified that in the caption.

> RE.4. Q1

MASE IR describes a model at the module level(now clarified in Table-1).
We traverse the graph and partition the graph into a set of subgraphs. Each subgraph must be able to fit on a single device, and the communication bandwidth between subgraphs must be below the maximum cross-device communication bandwidth. We've added a detailed explanation in Section-4.3.2-3).

> RE.5 Q2

Yes. Both software and hardware optimisations are performed in MASE IR, where they can be interleaved. MLIR is an exit point of the MASE flow. We've clarified that in Section 4.3.1.

> RE.6 Q3

Sorry about the confusion. Both Figure-8(was Figure-9) and Figure-9(was Figure-10) show the results of integer mixed-precision quantisation search. Figure-8 shows that with pure software-based quantisation, designs with minimal loss could have poor hardware energy efficiency. Figure-9 shows a fine-grain view of final precisions in the best hardware design across the layers. MASE can customise the precision of *every operation* and optimise the hardware at scale. We've improved Figure-9 and updated the captions for both figures to avoid confusion.