Verification and Validation in CFD

Welcome to this week’s edition of my newsletter — CFD Dose. In this story, we will explore together the crucial (but usually forgotten) concepts of verification and validation in CFD and their significance in ensuring the accuracy and reliability of our simulations.

Introduction

With the remarkable advancements in CPU performance and the ever-growing utilization of GPUs in numerical methods, CFD is becoming an very powerful tool for analyzing fluid flow phenomena in various industries. However, two of the most underrated and yet essential steps in performing a CFD simulation are verification and validation (V&V). These processes are often overlooked, but they are crucial in ensuring the reliability and accuracy of our CFD results. Without proper verification and validation, CFD simulations may lack meaning and could lead to false conclusions. In this edition, I will shed some light on the importance of these often-neglected steps and explore how they contribute to enhancing the credibility of our CFD analyses.

While these terms are often used mistakingly interchangeably, they have different meanings and distinct objectives!

What is really meant by verification and validation?

Verification refers to the process of assessing whether a computational model, mathematical formulation, or numerical method accurately represents the developers implementation of the model. It focuses on evaluating the consistency and correctness of the CFD software itself. Through verification, we gain confidence in the accuracy of the algorithms implemented in the software, ensuring that they correctly capture the mathematical principles governing fluid flow. [This is pretty much needed every time you implement a new block in your CFD code, checking that there are no programming errors.]

Verification is: “the process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model.” (AIAA-G-077–1998).

On the other hand, Validation aims to determine the degree to which a computational model can replicate the real-world physical behavior it intends to simulate. It involves comparing the simulation results with experimental or reference data obtained from physical experiments or reliable sources. Validation provides us with evidence that the CFD simulations can accurately predict the behavior of fluid flow in real-life scenarios, giving us confidence in their applicability and usefulness. [Mainly to check that CFD results agree with physical experiments (to a certain extent)]

Validation is: “The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model.” (AIAA-G-077–1998).

V&V are integral parts of the CFD simulation process, as they enable us to assess the credibility of our computational models and the reliability of our predictions. By constantly subjecting yout simulations to these rigorous processes, you can minimize errors, reduce uncertainties, and improve the overall quality of our CFD analyses — by a FAR margin, here’s why:

Importance of V&V

It can sound very intuitive that V&V are very important for any CFD simulation, and yet many people may forget/ overlook these steps while conducting a CFD study. For this, it is highly recommended to incorporate V&V in your framwork as an essential step before judging any results. Here are the main reasons why:

1. Ensuring Accuracy: V&V processes are crucial in minimizing errors and uncertainties in CFD simulations. Verification focuses on evaluating the correctness and consistency of the CFD software itself, including the implementation of mathematical models, numerical methods, and algorithms. By thoroughly verifying the software, we can identify and rectify any potential inaccuracies, ensuring that the CFD solver performs as intended and provides accurate results.
2. Enhancing Reliability: Validation, on the other hand, aims to assess the reliability of the CFD simulations by comparing the results with experimental or reference data. This step helps us determine the level of agreement between the simulated and real-world phenomena. By validating our simulations against reliable experimental data, we gain confidence in the predictive capabilities of the CFD models and their ability to capture the complex fluid behavior accurately.
3. Minimizing Risks: V&V provide an essential framework for minimizing risks associated with CFD simulations. In industries where safety, performance, and efficiency are critical; for example in aerospace-related applications, relying on validated CFD simulations becomes imperative. By subjecting your simulations to rigorous V&V, you can identify potential flaws (sometimes before even performing the CFD simulation), correct them, and make informed decisions based on reliable results, ultimately reducing risks and ensuring the integrity of the analyzed problem.
4. Optimizing Design and Performance: Verification and validation enable us to optimize the design and performance of systems or products by providing valuable insights into fluid behavior. Validated CFD simulations allow engineers to explore different design variations virtually, saving time and resources that would otherwise be required for physical prototyping and testing. By accurately predicting flow characteristics, CFD simulations can contribute to the development of more efficient and optimized designs.
5. Gaining Confidence and Trust: In academia more than in industry, V&V are essential for gaining confidence and establishing trust in the CFD results. By adhering to V&V standards, we could easily demonstrate the credibility of our simulations to peers, clients, and stakeholders. This fosters trust in the accuracy and reliability of CFD as a powerful tool for decision-making and problem-solving in various industries.

Verification Methods in CFD

As we mentioned earlier, verification methods in CFD focus on evaluating the correctness and accuracy of the software itself, including the implementation of mathematical models, numerical methods, and algorithms. Let’s explore some of the commonly used verification methods in CFD:

1. Method of Manufactured Solutions (MMS): MMS involves constructing analytical solutions with known properties and then solving them using the CFD code under consideration. By comparing the numerical results with the known analytical solutions, we can assess the accuracy and convergence properties of the code. MMS is particularly useful in identifying coding errors, verifying the implementation of mathematical models, and assessing the overall performance of the software [1–4].
2. Grid Convergence Study: A grid convergence (or grid dependency) test involves systematically refining the computational mesh and examining the spatial convergence of the numerical solution . By comparing the results obtained from different grid resolutions, we can determine the level of numerical error and convergence order. Grid convergence studies help ensure that the CFD simulations (expecially RANS and explicit LES derived methods) are not heavily influenced by the mesh resolution and provide reliable results regardless of the grid size [5].
3. Exact Solutions: In some cases, exact solutions exist for simplified CFD problems. By comparing the numerical results with these exact solutions, we can verify the accuracy of the numerical method and assess the influence of numerical errors. The method of exact solutions helps identify potential issues in the discretization schemes, boundary conditions, and solver settings.
4. Consistency Checks: Consistency checks involve examining the consistency of numerical solutions by comparing results obtained with different numerical methods, discretization schemes, or time-stepping algorithms. In this sense, consistency checks help ensure that the code implementation is consistent with the underlying mathematical formulation, and identify discrepancies or errors arising from numerical approximations.

Validation Methods in CFD

Validation methods in CFD aim to assess the reliability and predictive capabilities of the simulations by comparing the results with experimental or reference data. Let’s explore some of the common validation methods in CFD:

1. Experimental Data Comparison: Comparing CFD results with experimental data is a fundamental validation approach. This involves conducting physical experiments or utilizing reliable experimental data obtained from literature or trusted sources. By comparing the simulated quantities (e.g., statistics of velocities, pressure or temperature) with the corresponding experimental measurements, one can assess the agreement between the simulation and reality.
2. Benchmark Solutions: Benchmark solutions refer to well-defined CFD problems for which reference DNS or well-resolved LES solutions are available. These benchmark cases are often used for testing and validating CFD codes in the absence of experimental data for a given configuration. By comparing the numerical results with the known benchmark solutions, we can evaluate the accuracy and reliability of the CFD simulations.
3. Sensitivity Analysis: Sensitivity analysis involves varying input parameters and boundary conditions within their specified ranges and examining the resulting changes in the CFD predictions. By systematically analyzing the sensitivity of the simulations to various factors, we can gain insights into the robustness and reliability of the model. Such analysis usually helps identify parameters that significantly influence the results and assess the accuracy of the predictions under different operating conditions. [some CFD practitioners consider this as a verification method to adhere with the standard terminology of V&V].
4. Quantitative Metrics: Quantitative metrics, such as error norms (e.g., L1, L2, L∞ norms), root-mean-square error (RMSE), or coefficient of determination (R²), can be used to quantitatively compare the simulated results with the experimental or reference data. These metrics provide a numerical measure of the agreement between the simulation and the validated data, enabling a quantitative assessment of the simulation’s accuracy [6].

Case Studies: Verification and Validation of LPT Algorithm in T-Flows Code

To have a taste of what all that means, let’s explore together two case studies through which I demonstrated the verification and validation of an implemented Lagrangian Particle Tracking (LPT) algorithm in the T-Flows CFD code. This work was done during my PhD as a basis for testing hybrid RANS/LES on particle-laden flows. Let’s delve into the details of each case study:

1. Case Study 1: T-Junction Configuration

In the first case study, I focused on the verification of the implemented LPT algorithm using a rectangular T-junction configuration. The T-junction represents a common flow geometry encountered in many industrial applications. The primary objectives of this case was to verify the particle tracking algorithm on a serial version and prepare for parallelization.

The verification process involves comparing the simulated trajectories of particles within the T-junction with the corresponding solutions from ANSYS Fluent. By utilizing an identical mesh and problem setup for a laminar carrier flow, I could assess the correctness of the particle tracks from the lack of thereof with the algorithm’s implementation.

Once verified, the LPT algorithm was ready to be validated in a 90-degree bend configuration where V&V data were available in both laminar and turbulent regimes.

2. Case Study 2: 90-Degree Bend Configuration

In this second case study, I applied the verification and validation of the LPT algorithm using a 90-degree bend configuration; which entails a relatively more complex flow dynamics [6]. 90-degree bends are prevalent in various fluid flow systems, and accurately predicting particle behavior within such geometries is crucial for optimizing system performance and preventing particle accumulation or blockages.

In this case, the verification process was more stringent to ensure the acceptance criterion iss sufficient to move on (i.e. that the implemented algorithm is sound). The verification step involved a) plotting the the mean flow field at differnt locations in a code-to-code comparison, then b) comparing particle depostion efficiency curve obtained by T-Flows to the corresponding one from Fluent. This latter step ensures that the algorithm is not only implemented correctly but also that it produces accurate results within the bend configuration.

Once verified, the LPT algorithm was validated against experimental data acquired from particle tracking experiments from literature.

Through these thorough verification and validation steps, I could confidently utilize the LPT algorithm to track particles within complex flow configurations, providing valuable insights for a range of industrial applications. In the following, I’ll list some of the Best Practice Guidelines (PBGs) which should be followed when conducting V&V in CFD simulations.

Best Practices for V&V

To achieve meaningful and trustworthy results, it is essential to follow best practices throughout the verification and validation processes. Here are some of these:

1. Clearly define objectives and acceptance criteria: Before initiating the verification and validation processes, clearly define the objectives of the study and the acceptance criteria for V&V. This includes specifying the variables of interest, the expected accuracy, and the level of agreement with experimental or reference data. Defining these criteria helps guide the verification and validation activities and ensures that the simulations are aligned with the desired outcomes.
2. Establish proper documentation: Maintaining a thorough documentation of the V&V process is essential. Document the steps performed, assumptions made, methodologies employed, and the results obtained. This documentation provides transparency, facilitates reproducibility, and allows others to assess the credibility of the simulations. It is also valuable for future reference and for validating or updating the simulations as new data or methodologies become available.
3. Use multiple methods: Employing multiple V&V methods is highly recommended! Relying solely on one method may introduce bias or limitations. Contrarily, by using a combination of approaches, such as analytical solutions, benchmark cases, experimental comparisons, and sensitivity analysis, you can strengthen the credibility and confidence in the simulations. Each method offers unique insights and helps identify different aspects of the simulation’s accuracy and reliability.
4. Collaboration and peer review: Engage in collaboration and seek peer review throughout your V&V framework. Sharing your work with experts in the field and receiving feedback can provide valuable insights, identify potential errors, and ensure that the methodologies and results are sound. Collaboration fosters knowledge exchange, encourages good practices, and strengthens the overall credibility of the work. This has been a game-changer for me — sometimes you feel your model is behaving perfectly when it is exactly on the other side of the spectrum (just a couple of errors cancelling each other out). So be very careful with the confirmation bias! ☝🏼

Summary

In summary, verification (solving the equations right) and validation (solving the right equations) are vital steps in any CFD simulation. They ensure accuracy, enhance reliability, minimize risks, optimize designs, and contribute to building confidence and trust in the results. By investing time and effort into V&V, you will save a lot of time down the line in your CFD journey, and will be able make informed decisions based on the obtained results. The other way round, if you rush/ or neglect these steps, it will always come back in a much more time “lost”.

I hope that I could give you a good insight into how to enhance the credibility and reliability of your CFD simulations. 💻 🌊

The boring stuff — but they matter!

• If you want to support this blog, feel free to give a round of 👏 (50x+ makes my story get featured) and share the blog with your ⭕️. Also, feel free to follow me on Medium so you get all the updates from this blog.
• If you feel lost and wondering where to start your CFD journey, let’s have a 1:1 free session and discover each other’s stories — maybe I could be of a little help.

References

[1] K. Salari and P. Knupp. “Code verification by the Method of Manufactured Solutions”. Technical report, Sandia National Lab., USA, SAND2000–1444, 2000.

[2] P. J. Roache, “Code Verification by the Method of Manufactured Solutions,” J. Fluids Eng., vol. 124, no. 1, p. 4, 2002.

[3] P. J. Roache, Verification and validation in computational science and engineering, vol. 895. Hermosa Albuquerque, NM, 1998.

[4] A. Hodaib, Influence of radiative effects on buoyancy-induced flows in high-pressure compressor inter-disk cavities, PhD thesis, 2022 — University of Aix Marseille.

[5] Sayed, M. A., Dehbi, A., Niceno, B., & Mikityuk, K. (2021-b). Particle subgrid scale modeling in hubrid rans/les of turbulent channel flow at low-to-moderate reynolds number. J. Powder Technology. https://doi.org/https://doi.org/10.1016/j.powtec.2021.11.057.

[6] Sayed, M. A., Dehbi, A., Niceno, B., & Mikityuk, K. (2021-b). Particle subgrid scale modeling in hubrid rans/les of turbulent channel flow at low-to-moderate reynolds number. J. Powder Technology. https://doi.org/https://doi.org/10.1016/j.powtec.2021.11.057.