This work aims to provide a numerical treatment to deal with PDEs on surfaces with singularity al... more This work aims to provide a numerical treatment to deal with PDEs on surfaces with singularity along smooth curves. Previously, we proposed a low-order localized meshless method for discretizing the surface Laplacian operator on surfaces with folded. It is observed that the previously proposed strategy of approximating surface Laplacian at fold does not work with a recently developed high-order embedding meshless method for smooth surface. In this paper, we propose using the graph Laplacian to handel the surface singularity and study how it can be used to solve PDEs on folded surfaces.
WIT transactions on engineering sciences, Sep 10, 2019
We present three recently proposed kernel-based collocation methods in unified notations as an ea... more We present three recently proposed kernel-based collocation methods in unified notations as an easy reference for practitioners who need to solve PDEs on surfaces S ⊂ R d. These PDEs closely resemble their Euclidean counterparts, except that the problem domains change from bulk regions with a flat geometry of some surfaces, on which curvatures play an important role in the physical processes. First, we present a formulation to solve surface PDEs in a narrow band domain containing the surface. This class of numerical methods is known as the embedding types. Next, we present another formulation that works solely on the surface, which is commonly referred to as the intrinsic approach. Convergent estimates and numerical examples for both formulations will be given. For the latter, we solve both the linear and nonlinear time-dependent parabolic equations on static and moving surfaces. Keywords: kernel-based collocation methods, elliptic partial differential equations on manifolds, convergence estimate. for some surface differential operator L S : H m (S) → H m−2 (S) with W m ∞ (S)-bounded coefficients a S , c S : S → R, and b S : S → R d. We assume the existence [2] of classical solutions u * S to (2) in Hilbert spaces H m (S). 2 EMBEDDING KANSA METHODS Methods in this section are variants of Kansa methods [3], [4] that built upon some constantalong-normal property. They are generalization of the finite difference based closest point method [5] and its meshfree extension [6]. Firstly, we define the closest point mapping cp cp(x) = arg inf ξ∈S ξ − x 2 (R d) ,
Partial differential equations (PDEs) on surfaces arise in a variety of application areas includi... more Partial differential equations (PDEs) on surfaces arise in a variety of application areas including biological systems, medical imaging, fluid dynamics, mathematical physics, image processing and computer graphics. In this paper, we propose a radial basis function (RBF) discretization of the closest point method. The corresponding localized meshless method may be used to approximate diffusion on smooth or folded surfaces. Our method has the benefit of having an a priori error bound in terms of percentage of the norm of the solution. A stable solver is used to avoid the ill-conditioning that arises when the radial basis functions (RBFs) become flat.
There are plenty of applications and analysis for time-independent elliptic partial differential ... more There are plenty of applications and analysis for time-independent elliptic partial differential equations in the literature hinting at the benefits of overtesting by using more collocation conditions than the number of basis functions. Overtesting not only reduces the problem size, but is also known to be necessary for stability and convergence of widely used unsymmetric Kansa-type strong-form collocation methods. We consider kernelbased meshfree methods, which is a method of lines with collocation and overtesting spatially, for solving parabolic partial differential equations on surfaces without parametrization. In this paper, we extend the time-independent convergence theories for overtesting techniques to the parabolic equations on smooth and closed surfaces.
The aim of this paper is to present partial differential equations (PDEs) on surface to the commu... more The aim of this paper is to present partial differential equations (PDEs) on surface to the community of methods of fundamental solutions (MFS). First, we present an embedding formulation to embed surface PDEs into a domain so that MFS can be applied after the PDEs is homogenized with a particular solution. Next, we discuss how the domain-MFS method can be used to directly collocate surface PDEs. Some numerical demonstrations were included to study the effect of basis functions and source point locations.
The strong-form asymmetric kernel-based collocation method, commonly referred to as the Kansa met... more The strong-form asymmetric kernel-based collocation method, commonly referred to as the Kansa method, is easy to implement and hence is widely used for solving engineering problems and partial differential equations despite the lack of theoretical support. The simple leastsquares (LS) formulation, on the other hand, makes the study of its solvability and convergence rather nontrivial. In this paper, we focus on general second order linear elliptic differential equations in Ω ⊂ R d under Dirichlet boundary conditions. With kernels that reproduce H m (Ω) and some smoothness assumptions on the solution, we provide denseness conditions for a constrained leastsquares method and a class of weighted least-squares algorithms to be convergent. Theoretically, we identify some H 2 (Ω) convergent LS formulations that have an optimal error behavior like h m−2. We also demonstrate the effects of various collocation settings on the respective convergence rates, as well as how these formulations perform with high order kernels and when coupled with the stable evaluation technique for the Gaussian kernel.
We analyze a least-squares strong-form kernel collocation formulation for solving second order el... more We analyze a least-squares strong-form kernel collocation formulation for solving second order elliptic PDEs on smooth, connected and compact surfaces with bounded geometry. The methods do not require any partial derivatives of surface normal vectors or metric. Based on some standard smoothness assumptions for high order convergence, we provide the sufficient denseness conditions on the collocation points to ensure the methods are convergent. Besides of some convergence verifications, we also simulate some reaction-diffusion equations to exhibit the pattern formations.
We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov reg... more We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov regularization with W m 2-penalty is considered. We provide error estimates that are comparable to kernel-based interpolation in cases which the function it is approximating is within and is outside of the native space of the kernel. Our proven theories concern the denseness condition of collocation points and selection of regularization parameters. In particular, the unregularized least-squares method is shown to have W μ 2 () convergence for μ > d /2 on smooth domain ⊂ R d. For any properly regularized least-squares method, the same convergence estimates hold for a large range of μ ≥ 0. These results are extended to the case of noisy data. Numerical demonstrations are provided to verify the theoretical results. In terms of applications, we also apply the proposed method to solve a heat equation whose initial condition has huge oscillation in the domain.
There are plenty of applications and analysis for time-independent elliptic partial differential ... more There are plenty of applications and analysis for time-independent elliptic partial differential equations in the literature hinting at the benefits of overtesting by using more collocation conditions than the number of basis functions. Overtesting not only reduces the problem size, but is also known to be necessary for stability and convergence of widely used unsymmetric Kansa-type strong-form collocation methods. We consider kernelbased meshfree methods, which is a method of lines with collocation and overtesting spatially, for solving parabolic partial differential equations on surfaces without parametrization. In this paper, we extend the time-independent convergence theories for overtesting techniques to the parabolic equations on smooth and closed surfaces.
We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov reg... more We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov regularization with W m 2-penalty is considered. We provide error estimates that are comparable to kernel-based interpolation in cases which the function it is approximating is within and is outside of the native space of the kernel. Our proven theories concern the denseness condition of collocation points and selection of regularization parameters. In particular, the unregularized least-squares method is shown to have W μ 2 () convergence for μ > d /2 on smooth domain ⊂ R d. For any properly regularized least-squares method, the same convergence estimates hold for a large range of μ ≥ 0. These results are extended to the case of noisy data. Numerical demonstrations are provided to verify the theoretical results. In terms of applications, we also apply the proposed method to solve a heat equation whose initial condition has huge oscillation in the domain.
Despite the recent success of deep learning models for text generation, generating clinically acc... more Despite the recent success of deep learning models for text generation, generating clinically accurate reports remains challenging. More precisely modeling the relationships of the abnormalities revealed in an X-ray image has been found promising to enhance the clinical accuracy. In this paper, we first introduce a novel knowledge graph structure called an attributed abnormality graph (ATAG). It consists of interconnected abnormality nodes and attribute nodes for better capturing more fine-grained abnormality details. In contrast to the existing methods where the abnormality graph are constructed manually, we propose a methodology to automatically construct the fine-grained graph structure based on annotated X-ray reports and the RadLex radiology lexicon. We then learn the ATAG embeddings as part of a deep model with an encoder-decoder architecture for the report generation. In particular, graph attention networks are explored to encode the relationships among the abnormalities and their attributes. A hierarchical attention attention and a gating mechanism are specifically designed to further enhance the generation quality. We carry out extensive experiments based on the benchmark datasets, and show that the proposed ATAG-based deep model outperforms the SOTA methods by a large margin in ensuring the clinical accuracy of the generated reports.
We develop a deep learning approach to extract ray directions at discrete locations by analyzing ... more We develop a deep learning approach to extract ray directions at discrete locations by analyzing highly oscillatory wave fields. A deep neural network is trained on a set of local planewave fields to predict ray directions at discrete locations. The resulting deep neural network is then applied to a reduced-frequency Helmholtz solution to extract the directions, which are further incorporated into a ray-based interior-penalty discontinuous Galerkin (IPDG) method to solve the Helmholtz equations at higher frequencies. In this way, we observe no apparent pollution effects in the resulting Helmholtz solutions in inhomogeneous media. Our 2D and 3D numerical results show that the proposed scheme is very efficient and yields highly accurate solutions.
International Journal of Computational Methods and Experimental Measurements, 2017
In this paper, we apply the recently proposed fast block-greedy algorithm to a convergent kernel-... more In this paper, we apply the recently proposed fast block-greedy algorithm to a convergent kernel-based collocation method. In particular, we discretize three-dimensional second-order elliptic differential equations by the meshless asymmetric collocation method with over-sampling. Approximated solutions are obtained by solving the resulting weighted least squares problem. Such formulation has been proven to have optimal convergence in H 2. Our aim is to investigate the convergence behaviour of some three dimensional test problems. We also study the low-rank solution by restricting the approximation in some smaller trial subspaces. A block-greedy algorithm, which costs at most O(NK 2) to select K columns (or trial centers) out of an M × N overdetermined matrix, is employed for such an adaptivity. Numerical simulations are provided to justify these reductions.
Partial differential equations (PDEs) on surfaces arise in a variety of application areas includi... more Partial differential equations (PDEs) on surfaces arise in a variety of application areas including biological systems, medical imaging, fluid dynamics, mathematical physics, image processing and computer graphics. In this paper, we propose a radial basis function (RBF) discretization of the closest point method. The corresponding localized meshless method may be used to approximate diffusion on smooth or folded surfaces. Our method has the benefit of having an a priori error bound in terms of percentage of the norm of the solution. A stable solver is used to avoid the ill-conditioning that arises when the radial basis functions (RBFs) become flat.
Advances in Trefftz Methods and Their Applications, 2020
The aim of this paper is to present partial differential equations (PDEs) on surface to the commu... more The aim of this paper is to present partial differential equations (PDEs) on surface to the community of methods of fundamental solutions (MFS). First, we present an embedding formulation to embed surface PDEs into a domain so that MFS can be applied after the PDEs is homogenized with a particular solution. Next, we discuss how the domain-MFS method can be used to directly collocate surface PDEs. Some numerical demonstrations were included to study the effect of basis functions and source point locations. 1. Partial differential equations on surfaces. In this paper, we focus on second-order elliptic partial differential equations (PDEs) posed on some sufficiently smooth, connected, and compact surface S ⊂ R with bounded geometry. Without loss of generality, we assume dim(S) = d − 1, a.k.a., S has co-dimension 1. We denote the unit outward normal vector at x ∈ S as n = n(x) and the corresponding projection matrix to the tangent space of S at x as P(x) = [~ P1, . . . , ~ Pd](x) := Id ...
Distributed stochastic gradient descent (SGD) algorithms are widely deployed in training large-sc... more Distributed stochastic gradient descent (SGD) algorithms are widely deployed in training large-scale deep learning models, while the communication overhead among workers becomes the new system bottleneck. Recently proposed gradient sparsification techniques, especially Top-$k$ sparsification with error compensation (TopK-SGD), can significantly reduce the communication traffic without an obvious impact on the model accuracy. Some theoretical studies have been carried out to analyze the convergence property of TopK-SGD. However, existing studies do not dive into the details of Top-$k$ operator in gradient sparsification and use relaxed bounds (e.g., exact bound of Random-$k$) for analysis; hence the derived results cannot well describe the real convergence performance of TopK-SGD. To this end, we first study the gradient distributions of TopK-SGD during the training process through extensive experiments. We then theoretically derive a tighter bound for the Top-$k$ operator. Finally, ...
We analyze a least-squares strong-form kernel collocation formulation for solving second order el... more We analyze a least-squares strong-form kernel collocation formulation for solving second order elliptic PDEs on smooth, connected and compact surfaces with bounded geometry. The methods do not require any partial derivatives of surface normal vectors or metric. Based on some standard smoothness assumptions for high order convergence, we provide the sufficient denseness conditions on the collocation points to ensure the methods are convergent. Besides of some convergence verifications, we also simulate some reaction-diffusion equations to exhibit the pattern formations.
Partial differential equations (PDEs) on surfaces arise in a variety of application areas includi... more Partial differential equations (PDEs) on surfaces arise in a variety of application areas including biological systems, medical imaging, fluid dynamics, mathematical physics, image processing and computer graphics. In this paper, we propose a radial basis function (RBF) discretization of the closest point method. The corresponding localized meshless method may be used to approximate diffusion on smooth or folded surfaces. Our method has the benefit of having an a priori error bound in terms of percentage of the norm of the solution. A stable solver is used to avoid the ill-conditioning that arises when the radial basis functions (RBFs) become flat.
This work aims to provide a numerical treatment to deal with PDEs on surfaces with singularity al... more This work aims to provide a numerical treatment to deal with PDEs on surfaces with singularity along smooth curves. Previously, we proposed a low-order localized meshless method for discretizing the surface Laplacian operator on surfaces with folded. It is observed that the previously proposed strategy of approximating surface Laplacian at fold does not work with a recently developed high-order embedding meshless method for smooth surface. In this paper, we propose using the graph Laplacian to handel the surface singularity and study how it can be used to solve PDEs on folded surfaces.
WIT transactions on engineering sciences, Sep 10, 2019
We present three recently proposed kernel-based collocation methods in unified notations as an ea... more We present three recently proposed kernel-based collocation methods in unified notations as an easy reference for practitioners who need to solve PDEs on surfaces S ⊂ R d. These PDEs closely resemble their Euclidean counterparts, except that the problem domains change from bulk regions with a flat geometry of some surfaces, on which curvatures play an important role in the physical processes. First, we present a formulation to solve surface PDEs in a narrow band domain containing the surface. This class of numerical methods is known as the embedding types. Next, we present another formulation that works solely on the surface, which is commonly referred to as the intrinsic approach. Convergent estimates and numerical examples for both formulations will be given. For the latter, we solve both the linear and nonlinear time-dependent parabolic equations on static and moving surfaces. Keywords: kernel-based collocation methods, elliptic partial differential equations on manifolds, convergence estimate. for some surface differential operator L S : H m (S) → H m−2 (S) with W m ∞ (S)-bounded coefficients a S , c S : S → R, and b S : S → R d. We assume the existence [2] of classical solutions u * S to (2) in Hilbert spaces H m (S). 2 EMBEDDING KANSA METHODS Methods in this section are variants of Kansa methods [3], [4] that built upon some constantalong-normal property. They are generalization of the finite difference based closest point method [5] and its meshfree extension [6]. Firstly, we define the closest point mapping cp cp(x) = arg inf ξ∈S ξ − x 2 (R d) ,
Partial differential equations (PDEs) on surfaces arise in a variety of application areas includi... more Partial differential equations (PDEs) on surfaces arise in a variety of application areas including biological systems, medical imaging, fluid dynamics, mathematical physics, image processing and computer graphics. In this paper, we propose a radial basis function (RBF) discretization of the closest point method. The corresponding localized meshless method may be used to approximate diffusion on smooth or folded surfaces. Our method has the benefit of having an a priori error bound in terms of percentage of the norm of the solution. A stable solver is used to avoid the ill-conditioning that arises when the radial basis functions (RBFs) become flat.
There are plenty of applications and analysis for time-independent elliptic partial differential ... more There are plenty of applications and analysis for time-independent elliptic partial differential equations in the literature hinting at the benefits of overtesting by using more collocation conditions than the number of basis functions. Overtesting not only reduces the problem size, but is also known to be necessary for stability and convergence of widely used unsymmetric Kansa-type strong-form collocation methods. We consider kernelbased meshfree methods, which is a method of lines with collocation and overtesting spatially, for solving parabolic partial differential equations on surfaces without parametrization. In this paper, we extend the time-independent convergence theories for overtesting techniques to the parabolic equations on smooth and closed surfaces.
The aim of this paper is to present partial differential equations (PDEs) on surface to the commu... more The aim of this paper is to present partial differential equations (PDEs) on surface to the community of methods of fundamental solutions (MFS). First, we present an embedding formulation to embed surface PDEs into a domain so that MFS can be applied after the PDEs is homogenized with a particular solution. Next, we discuss how the domain-MFS method can be used to directly collocate surface PDEs. Some numerical demonstrations were included to study the effect of basis functions and source point locations.
The strong-form asymmetric kernel-based collocation method, commonly referred to as the Kansa met... more The strong-form asymmetric kernel-based collocation method, commonly referred to as the Kansa method, is easy to implement and hence is widely used for solving engineering problems and partial differential equations despite the lack of theoretical support. The simple leastsquares (LS) formulation, on the other hand, makes the study of its solvability and convergence rather nontrivial. In this paper, we focus on general second order linear elliptic differential equations in Ω ⊂ R d under Dirichlet boundary conditions. With kernels that reproduce H m (Ω) and some smoothness assumptions on the solution, we provide denseness conditions for a constrained leastsquares method and a class of weighted least-squares algorithms to be convergent. Theoretically, we identify some H 2 (Ω) convergent LS formulations that have an optimal error behavior like h m−2. We also demonstrate the effects of various collocation settings on the respective convergence rates, as well as how these formulations perform with high order kernels and when coupled with the stable evaluation technique for the Gaussian kernel.
We analyze a least-squares strong-form kernel collocation formulation for solving second order el... more We analyze a least-squares strong-form kernel collocation formulation for solving second order elliptic PDEs on smooth, connected and compact surfaces with bounded geometry. The methods do not require any partial derivatives of surface normal vectors or metric. Based on some standard smoothness assumptions for high order convergence, we provide the sufficient denseness conditions on the collocation points to ensure the methods are convergent. Besides of some convergence verifications, we also simulate some reaction-diffusion equations to exhibit the pattern formations.
We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov reg... more We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov regularization with W m 2-penalty is considered. We provide error estimates that are comparable to kernel-based interpolation in cases which the function it is approximating is within and is outside of the native space of the kernel. Our proven theories concern the denseness condition of collocation points and selection of regularization parameters. In particular, the unregularized least-squares method is shown to have W μ 2 () convergence for μ > d /2 on smooth domain ⊂ R d. For any properly regularized least-squares method, the same convergence estimates hold for a large range of μ ≥ 0. These results are extended to the case of noisy data. Numerical demonstrations are provided to verify the theoretical results. In terms of applications, we also apply the proposed method to solve a heat equation whose initial condition has huge oscillation in the domain.
There are plenty of applications and analysis for time-independent elliptic partial differential ... more There are plenty of applications and analysis for time-independent elliptic partial differential equations in the literature hinting at the benefits of overtesting by using more collocation conditions than the number of basis functions. Overtesting not only reduces the problem size, but is also known to be necessary for stability and convergence of widely used unsymmetric Kansa-type strong-form collocation methods. We consider kernelbased meshfree methods, which is a method of lines with collocation and overtesting spatially, for solving parabolic partial differential equations on surfaces without parametrization. In this paper, we extend the time-independent convergence theories for overtesting techniques to the parabolic equations on smooth and closed surfaces.
We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov reg... more We consider discrete least-squares methods using radial basis functions. A general 2-Tikhonov regularization with W m 2-penalty is considered. We provide error estimates that are comparable to kernel-based interpolation in cases which the function it is approximating is within and is outside of the native space of the kernel. Our proven theories concern the denseness condition of collocation points and selection of regularization parameters. In particular, the unregularized least-squares method is shown to have W μ 2 () convergence for μ > d /2 on smooth domain ⊂ R d. For any properly regularized least-squares method, the same convergence estimates hold for a large range of μ ≥ 0. These results are extended to the case of noisy data. Numerical demonstrations are provided to verify the theoretical results. In terms of applications, we also apply the proposed method to solve a heat equation whose initial condition has huge oscillation in the domain.
Despite the recent success of deep learning models for text generation, generating clinically acc... more Despite the recent success of deep learning models for text generation, generating clinically accurate reports remains challenging. More precisely modeling the relationships of the abnormalities revealed in an X-ray image has been found promising to enhance the clinical accuracy. In this paper, we first introduce a novel knowledge graph structure called an attributed abnormality graph (ATAG). It consists of interconnected abnormality nodes and attribute nodes for better capturing more fine-grained abnormality details. In contrast to the existing methods where the abnormality graph are constructed manually, we propose a methodology to automatically construct the fine-grained graph structure based on annotated X-ray reports and the RadLex radiology lexicon. We then learn the ATAG embeddings as part of a deep model with an encoder-decoder architecture for the report generation. In particular, graph attention networks are explored to encode the relationships among the abnormalities and their attributes. A hierarchical attention attention and a gating mechanism are specifically designed to further enhance the generation quality. We carry out extensive experiments based on the benchmark datasets, and show that the proposed ATAG-based deep model outperforms the SOTA methods by a large margin in ensuring the clinical accuracy of the generated reports.
We develop a deep learning approach to extract ray directions at discrete locations by analyzing ... more We develop a deep learning approach to extract ray directions at discrete locations by analyzing highly oscillatory wave fields. A deep neural network is trained on a set of local planewave fields to predict ray directions at discrete locations. The resulting deep neural network is then applied to a reduced-frequency Helmholtz solution to extract the directions, which are further incorporated into a ray-based interior-penalty discontinuous Galerkin (IPDG) method to solve the Helmholtz equations at higher frequencies. In this way, we observe no apparent pollution effects in the resulting Helmholtz solutions in inhomogeneous media. Our 2D and 3D numerical results show that the proposed scheme is very efficient and yields highly accurate solutions.
International Journal of Computational Methods and Experimental Measurements, 2017
In this paper, we apply the recently proposed fast block-greedy algorithm to a convergent kernel-... more In this paper, we apply the recently proposed fast block-greedy algorithm to a convergent kernel-based collocation method. In particular, we discretize three-dimensional second-order elliptic differential equations by the meshless asymmetric collocation method with over-sampling. Approximated solutions are obtained by solving the resulting weighted least squares problem. Such formulation has been proven to have optimal convergence in H 2. Our aim is to investigate the convergence behaviour of some three dimensional test problems. We also study the low-rank solution by restricting the approximation in some smaller trial subspaces. A block-greedy algorithm, which costs at most O(NK 2) to select K columns (or trial centers) out of an M × N overdetermined matrix, is employed for such an adaptivity. Numerical simulations are provided to justify these reductions.
Partial differential equations (PDEs) on surfaces arise in a variety of application areas includi... more Partial differential equations (PDEs) on surfaces arise in a variety of application areas including biological systems, medical imaging, fluid dynamics, mathematical physics, image processing and computer graphics. In this paper, we propose a radial basis function (RBF) discretization of the closest point method. The corresponding localized meshless method may be used to approximate diffusion on smooth or folded surfaces. Our method has the benefit of having an a priori error bound in terms of percentage of the norm of the solution. A stable solver is used to avoid the ill-conditioning that arises when the radial basis functions (RBFs) become flat.
Advances in Trefftz Methods and Their Applications, 2020
The aim of this paper is to present partial differential equations (PDEs) on surface to the commu... more The aim of this paper is to present partial differential equations (PDEs) on surface to the community of methods of fundamental solutions (MFS). First, we present an embedding formulation to embed surface PDEs into a domain so that MFS can be applied after the PDEs is homogenized with a particular solution. Next, we discuss how the domain-MFS method can be used to directly collocate surface PDEs. Some numerical demonstrations were included to study the effect of basis functions and source point locations. 1. Partial differential equations on surfaces. In this paper, we focus on second-order elliptic partial differential equations (PDEs) posed on some sufficiently smooth, connected, and compact surface S ⊂ R with bounded geometry. Without loss of generality, we assume dim(S) = d − 1, a.k.a., S has co-dimension 1. We denote the unit outward normal vector at x ∈ S as n = n(x) and the corresponding projection matrix to the tangent space of S at x as P(x) = [~ P1, . . . , ~ Pd](x) := Id ...
Distributed stochastic gradient descent (SGD) algorithms are widely deployed in training large-sc... more Distributed stochastic gradient descent (SGD) algorithms are widely deployed in training large-scale deep learning models, while the communication overhead among workers becomes the new system bottleneck. Recently proposed gradient sparsification techniques, especially Top-$k$ sparsification with error compensation (TopK-SGD), can significantly reduce the communication traffic without an obvious impact on the model accuracy. Some theoretical studies have been carried out to analyze the convergence property of TopK-SGD. However, existing studies do not dive into the details of Top-$k$ operator in gradient sparsification and use relaxed bounds (e.g., exact bound of Random-$k$) for analysis; hence the derived results cannot well describe the real convergence performance of TopK-SGD. To this end, we first study the gradient distributions of TopK-SGD during the training process through extensive experiments. We then theoretically derive a tighter bound for the Top-$k$ operator. Finally, ...
We analyze a least-squares strong-form kernel collocation formulation for solving second order el... more We analyze a least-squares strong-form kernel collocation formulation for solving second order elliptic PDEs on smooth, connected and compact surfaces with bounded geometry. The methods do not require any partial derivatives of surface normal vectors or metric. Based on some standard smoothness assumptions for high order convergence, we provide the sufficient denseness conditions on the collocation points to ensure the methods are convergent. Besides of some convergence verifications, we also simulate some reaction-diffusion equations to exhibit the pattern formations.
Partial differential equations (PDEs) on surfaces arise in a variety of application areas includi... more Partial differential equations (PDEs) on surfaces arise in a variety of application areas including biological systems, medical imaging, fluid dynamics, mathematical physics, image processing and computer graphics. In this paper, we propose a radial basis function (RBF) discretization of the closest point method. The corresponding localized meshless method may be used to approximate diffusion on smooth or folded surfaces. Our method has the benefit of having an a priori error bound in terms of percentage of the norm of the solution. A stable solver is used to avoid the ill-conditioning that arises when the radial basis functions (RBFs) become flat.
Uploads
Papers by Ka Chun Cheung