Multi-Fidelity Data-Driven Physics-Informed Neural Networks (MF-DD-PINNs)

<div class="container_jedd"> <h1>Multi-Fidelity Data-Driven Physics-Informed Neural Networks (MF-DD-PINNs)</h1> - Reference: [Yang et al., Data-driven physics-informed neural networks: A digital twin perspective, 2024.](https://arxiv.org/abs/2401.08667) - Post by: [Jedd Yang](https://hackmd.io/@jeddiot) - Date: 2025-May-13 (remastered on 2025-Dec-20) - Keywords: Computational Mechanics, Digital Twin, Neural Networks <h2>Background</h2> <div class="section-box"> <h3>Scenario</h3> The fluid digital twin system requires a seamless, real-time synchronization between physical assets and their virtual counterparts. While traditional Computational Fluid Dynamics (CFD) offers high accuracy, its heavy reliance on iterative solvers and complex meshing makes it too computationally expensive and slow for the instantaneous feedback loops required in DT environments. The PINN Alternative To overcome these latency barriers, Physics-Informed Neural Networks (PINNs) have emerged as a transformative surrogate modeling approach. By embedding physical laws (like the Navier-Stokes equations) directly into the neural network's loss function, PINNs offer three key advantages: - **Mesh-Free Architecture**: Eliminates the time-consuming process of manual grid generation, allowing for automated interaction between physical and virtual spaces. - **Near-Instantaneous Inference**: Once trained, the model can predict complex flow fields in milliseconds, enabling real-time monitoring and decision-making. - **Massive Parallelization**: Leverages GPU acceleration to handle high-dimensional parametric studies far more efficiently than CPU-based legacy solvers. --- <h3>Problem</h3> - The Reynolds Number ($Re$) Barrier: "Data-free" PINNs struggle as flow transitions to convection-dominant regimes where complex phenomena like vortex shedding begin. The loss landscape becomes too complex for the optimizer to find a global minimum without a "guide." - The Training Paradox (Data Dependency): Current solutions to the $Re$ barrier often involve Data-Driven PINNs (DD-PINNs). However, using high-fidelity CFD data to train these models defeats the purpose of creating a PINN, as the goal is to replace expensive CFD, not require it as a prerequisite. - Extrapolation \& Parametric Limitations: Standard PINNs are often non-parametric, meaning they are "hard-coded" for one specific $Re$. Even when data-driven PINNs succeed at a specific $Re$, they often exhibit poor extrapolation capabilities—failing to predict flow fields when the $Re$ is changed without complete retraining. --- <h3>Solution</h3> The authors propose a Multi-Fidelity Data-Driven PINN (MF-DD-PINN) framework designed to bridge the gap between theoretical "toy" problems and real-world Digital Twin (DT) applications. The solution consists of three key pillars: - Optimized Sampling Strategy: While investigating various adaptive sampling methods, the authors found that for DD-PINNs, Uniform Random Sampling is actually superior. It prevents the model from over-focusing on specific areas and balances the "local regularization" provided by the guide data. - Multi-Fidelity Data Integration: The framework is extended to leverage heterogeneous data sources. It combines: - Low-fidelity data: Dense but less accurate data (e.g., from quick numerical simulations). - High-fidelity data: Accurate but sparse data (e.g., from physical sensors in the real-world space). - Parametric Learning & UQ: Unlike standard models fixed to one Reynolds number, this framework solves Parametric Navier-Stokes equations. This allows for real-time prediction across a range of $Re$ and includes Uncertainty Quantification (UQ) via an ensemble approach, which tells the user how much to "trust" the prediction at any given $Re$. </div> <h2>Methodology</h2> <div class="section-box"> ### 1. Foundational Architecture: PINNs <center> <img src="https://hackmd.io/_uploads/Sy-sl4NQ-e.png" width="700"> <figcaption>Fig. 1: Overall architectures of PINN. The data-free PINN does not include data-driven loss term where as data-driven PINN does.</figcaption> <br> </center> The core of the methodology relies on Physics-Informed Neural Networks (PINNs), which treat the solution of Partial Differential Equations (PDEs) as an optimization problem. The network is trained to minimize a composite loss function: $$L_{PINN} = \underbrace{L_{mass} + L_{mommentum}}_{L_{residual}} + L_{BC} + L_{IC} \tag{1}$$ Unlike traditional CFD, PINNs are **mesh-free**; they evaluate these losses at discrete **collocation points**. The accuracy of the "data-free" approach depends entirely on how effectively these points cover regions of high physical complexity (e.g., boundary layers or high velocity gradients). --- ### 2. Proposed Feature: Adaptive Blending Sampling To improve "data-free" PINNs, the authors implement an iterative sampling strategy. Points are re-sampled periodically during training, focusing on regions where the model struggles. **The Blending Scheme:** Instead of relying on a single indicator, the paper proposes a hybrid approach using a **switching logic** to select points based on: 1. **PDE Residual:** Focuses on areas where the physics is not yet "obeyed." 2. **Gradient Magnitude:** Captures sharp changes in velocity or pressure fields. 3. **Vorticity:** Essential for capturing rotational structures in fluid dynamics. **Control Parameters:** The effectiveness is tuned via the **Update Ratio ($N/M$)**, which determines what percentage of the collocation points are replaced, and **Stochastic Intensity ($k$)**, which maintains a level of randomness to ensure global exploration of the domain. --- ### 3. Proposed Feature: Data-Driven & Multi-Fidelity (MF) The research identifies that even with adaptive sampling, data-free PINNs fail at high $Re$ because the loss landscape becomes too complex for standard optimizers. The solution is the **DD-PINN** framework: $$L_{DD-PINN} = L_{PINN} + w_{data} \cdot L_{data} \tag{2}$$ **The Multi-Fidelity Extension:** To make the model viable for Digital Twins, the authors extend this to a **Multi-Fidelity (MF)** approach. This allows the model to leverage heterogeneous data: * **Low-Fidelity Data:** Dense but less accurate data (e.g., from coarse-grid numerical simulations) to provide a global "guide." * **High-Fidelity Data:** Highly accurate but sparse data (e.g., from physical sensors or expensive CFD) to anchor the model in reality. --- ### 4. Scalability: Parametric Navier-Stokes The ultimate methodological goal is transitioning from a **point-solution** solver to a **parametric** solver. * **Traditional PINNs:** Usually trained to solve for a single, fixed Reynolds number. * **Proposed DD-PINN:** Designed to solve **Parametric Navier-Stokes equations**, where the Reynolds number ($Re$) is an **input parameter**. This allows the Digital Twin to perform **real-time inference** across a continuous range of $Re$ without the need to retrain the model for every new physical condition. </div> <h2>Result</h2> <div class="section-box"> <h3>Problem setup (Lid-driven cavity)</h3> {%preview https://github.com/jeddiot/staggered-grid-lid-driven-cavity %} <h3>Data-Driven Physics-Informed Neural Networks (DD-PINNs)</h3> <center> <img src="https://hackmd.io/_uploads/ByFkgEEXbx.png" width="700"> <figcaption>Fig. 2: Predicted y-velocity flow fields by DD-PINN for parametric NS equations. Note that only Re = 100, 1000, 3200 are trained and other Reynolds numbers are test cases. For three-column figures: (left column) DNS with 160 × 160 grids, (middle column) DD-PINN, (right column) error.</figcaption> <br> </center> <h3>Multi-Fidelity Data-Driven Physics-Informed Neural Networks (MF-DD-PINNs)</h3> <center> <img src="https://hackmd.io/_uploads/ByLExVNXZe.png" width="700"> <figcaption>Fig. 3: Predicted y-velocity flow fields by MF-DD-PINN case 4 for parametric NS equations. Note that only Re = 100, 1000, and 3200 are trained and other Reynolds numbers are test cases. For three-column figures: (left column) DNS with 160 × 160 grids, (middle column) MF-DD-PINN case 4, (right column) error.</figcaption> <br> </center> </div> </div>