Physics-informed neural networks (PINNs) are known to suffer from optimization difficulty.In this work, we reveal the connection between the optimization difficulty of PINNs and activation functions.Specifically, we show that PINNs exhibit high sensitivity to activation functions when solving PDEs with distinct properties.Existing works usually choose activation functions by inefficient trial-and-error.To avoid the inefficient manual selection and to alleviate the optimization difficulty of PINNs, we introduce adaptive activation functions to search for the optimal function when solving different problems.We compare different adaptive activation functions and discuss their limitations in the context of PINNs.Furthermore, we propose to tailor the idea of learning combinations of candidate activation functions to the PINNs optimization, which has a higher requirement for the smoothness and diversity on learned functions.This is achieved by removing activation functions which cannot provide higher-order derivatives from the candidate set and incorporating elementary functions with different properties according to our prior knowledge about the PDE at hand.We further enhance the search space with adaptive slopes.The proposed adaptive activation function can be used to solve different PDE systems in an interpretable way.Its effectiveness is demonstrated on a series of benchmarks.Code is available at https://github.com/LeapLabTHU/AdaAFforPINNs.

Honghui Wang Honghui Wang

Lu Lu Lu Lu

Shiji Song Shiji Song

Gao Huang Gao Huang

Communications in Computational Physics

COMMUNICATIONS IN COMPUTATIONAL PHYSICS (CICP) 发表在物理问题的计算建模方面具有高科学价值的原创研究和调查论文。将展示所有物理科学中多物理和多尺度创新计算方法和建模的结果。

Communications in Computational Physics (CICP) publishes original research and survey papers of high scientific value in the field of computational modeling of physical problems. The journal showcases results of innovative computational methods and modeling across multiple physics disciplines, emphasizing multiphysics and multiscale approaches.

《Communications in Computational Physics》发表具有高科学价值的原创研究和调查论文，专注于物理问题的计算建模。期刊展示了在所有物理科学领域中，多物理和多尺度创新计算方法及建模的研究成果。

Learning Specialized Activation Functions for Physics-Informed Neural Networks

This work reveals the connection between the optimization difficulty of PINNs and activation functions and proposes to tailor the idea of learning combinations of candidate activation functions to the PINNs optimization, which has a higher requirement for the smoothness and diversity on learned functions.

Category	Quartile
物理与天体物理	3
物理与天体物理, 物理数学物理	4

Learning Specialized Activation Functions for Physics-Informed Neural Networks

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info

Category	Quartile
PHYSICS, MATHEMATICAL	1