---
# System prepended metadata

title: Untitled

---


We're really grateful to the reviewers for their careful reviews and helpful feedback.

We begin by addressing some concerns raised by multiple reviewers, and then respond to each reviewer in turn.

### Common Concerns

> Comparison with related work

Many thanks to Reviewers B and D for pointing us to more related works.

Typing systems, as pointed out by Reviewer B, automatically check a set of properties, allowing a legal user program to get those properties "for free", thus avoiding separate tests. However, this also limits the generality of user programs. For example, mapping a routing algorithm, which contains complex control flows, onto such typing systems is challenging. PEDIA is designed for general applications and employs tests to capture complex hardware behaviors, such as input-dependent behaviors with variable iteration latencies, that are challenging to be statically analyzed. To address this, PEDIA generates random input arrivals for the hardware components, both internally and externally, to simulate dynamic hardware behaviors with variable latencies. A challenge is that these random input arrivals may be redundant. To overcome this, PEDIA reduces the set of random arrivals to avoid redundancy using static analysis.

Software testing techniques, as mentioned by Reviewer D, have advanced in capturing bugs in concurrency contexts. However, these techniques cannot be directly applied to hardware testing due to significantly different specifications. For instance, while software focuses on execution orders, hardware requires cycle-accurate behavior. PEDIA provides an initial step towards bridging these two fields by offering a cycle-sensitive software interface. Exploring these existing software testing techniques in hardware testing presents challenges and could be an interesting avenue for future work.

In the revised draft, we will expand this discussion to two new background sections to Section 5 and compare PEDIA in detail with programming languages with hardware-related typing systems and existing software testing techniques.

> Bugs found by PEDIA 

PEDIA focuses on improving testing speed; however, we have also identified an interesting bug that could not be detected by existing HLS tools.

Particularly, a memory component named the load-store queue (LSQ) can cause deadlock if the program exhibits complex memory dependence. An LSQ contains a fixed-size memory buffer to store pending memory requests and checks their dependencies. Deadlock occurs when the buffer size is insufficient and cannot allocate enough memory requests to resolve dependencies. This bug can emerge in *any* HLS designs that contain LSQs, particularly those handling programs with complex memory dependencies. Existing approaches can only explore some cases by varying input values, while PEDIA efficiently identifies such bugs by strategically adding delays to the testing process.

In the revised draft, we will add a case study to Section 4 and explain how this bug is caused, why existing HLS tools cannot capture it, and how PEDIA captures it.

> Effects on test coverage and limitation

This is an interesting question. The coverage of the testing space remains unaffected by our method, as we specifically target the removal of only redundant test cases. Our approach uses static analysis based on the worst-case timing behaviors of the HLS design, given by the scheduling reports generated by HLS tools. These reports provide a safe over-approximation of timing behaviors, enabling us to achieve an effective reduction in redundant test cases without compromising coverage. It is worth noting that this method of over-approximation does not eliminate all redundant cases but has proven to be significantly effective.

A notable limitation arises when the scheduling analysis provided by existing HLS tools is unable to accurately estimate the worst-case timing for an HLS design. In such instances, PEDIA fails to reduce test cases. Still, this may affect testing time but does not affect coverage.

We will clarify that in Section 3.4.

---

### Reviewer A

> How significant are the technical contributions? What are the key insights? 

The cycle-sensitive user interface in software programming languages provided by PEDIA achieves efficient hardware testing for HLS. These contributions significantly reduce the need for human intervention in generating and optimizing hardware testing platforms. PEDIA not only enhances efficiency but also opens new possibilities for more complex and accurate testing scenarios without manual guidance. We will further clarify that in Section 1.

---

### Reviewer B

> PEDIA's interactive debugging feature works v.s. CPU-based simulators

In practice, debugging hardware designs can suffer from long feedback when errors occur late in execution, requiring the re-execution of all preceding states with CPU-based simulators. PEDIA introduces a solution through FPGA acceleration, enabling users to directly 'teleport' to the error state and continue debugging using CPU-based simulators. This approach significantly reduces the debugging feedback and streamlines the testing process. We will further clarify that in Section 3.5.

> Are there any specific types of HLS designs or applications where PEDIA's parallel hardware emulation approach is particularly beneficial or challenging?

PEDIA is amenable to HLS designs with unpredictable hardware behaviors, such as input-dependent control flows. These hardware behaviors are difficult to be formally verified without human guidance. However, when an HLS design is too large to fit onto a single FPGA device, PEDIA requires manual effort to partition the HLS design for separate tests, where the coverage is still preserved through static analysis. Automating the partitioning of HLS designs across multiple FPGA devices is one of our future works. We will further clarify that in Section 4.1.

> How does PEDIA's approach to data parallelism and data packing differ from traditional hardware emulation methods?

Traditional hardware emulation methods often rely on manual effort to make optimization decisions, including data parallelism and data packing. PEDIA automates this process with the proposed software interface. This interface orchestrates existing software or HLS passes, enabling efficient hardware emulation. We will further clarify that in Section 3.4.

> Can PEDIA be extended to support HLS tools other than Xilinx Vitis HLS, and what would be required for such an extension?

Yes. In our paper, we demonstrate PEDIA's compatibility with two HLS tools: AMD Xilinx Vitis HLS and Dynamatic from EPFL. While other HLS tools like Cadence Stratus HLS and the Intel HLS Compiler feature similar interfaces, they support different pragma formats. Integration with PEDIA for these tools can be achieved through the addition of pragma mapping. We will further clarify that in Section 3.1.

> What are the theoretical and practical limits of PEDIA's approach to HLS testing, and how might these be addressed in future work?


A theoretical limitation of PEDIA is its inability to eliminate all redundant test cases. Future work aims to develop a formal model that defines the coverage of an HLS design by considering both arrival times and input values. This future work could facilitate the application of more sophisticated software testing techniques, potentially reducing the testing space currently constrained by static analysis from HLS tools.

Practically, a significant limitation is the resource constraint of FPGA devices, which cannot fit large HLS designs. Currently, PEDIA requires manual effort for partitioning large designs for testing, because software analysis of an HLS design on hardware resources remains challenging. A future work is to integrate hardware resource estimation to enable automatic partitioning of HLS designs. This would allow for testing on multiple FPGA devices in parallel, improving PEDIA's scalability and efficiency.

We will add this discussion to Section 6.

---

### Reviewer C

> Is it possible to exploit the improved parallelism technique introduced in this paper in other existing systems?

Yes, sorry about the confusion. Capturing dynamic input arrivals and hardware parallelism for testing are two independent contributions, both achieved by the software interface proposed by PEDIA.

> scalability of PEDIA?

Thanks for your advice, we will compare PEDIA with all the baselines and add these results to Section 4.

---

### Reviewer D

> Real-world bugs

Thanks for your advice. A brief description of the bug is given in common concerns.
