In this thesis, we propose the idea of modifying "kernel" for inverse problems. According to the idea, we systematically investigate four kinds of classical inverse problems in mathematical physics: high order numerical differentiation; inverse heat conduction problem; Cauchy problem for Laplace equation; backward heat conduction problem.We analyze the ill-posedness of these inverse problems, i.e., the solution does not depend continuously on the data, and discuss their degree of ill-posedness. For computing these problems stably, we firstly analyze many regularization methods for a one-dimensional inverse heat conduction problem(1D IHCP) in the frequency space. We find an interesting relation among these methods and point out the natural cause of regularization. Consequently, we conclude an important property: all regularization methods for 1D IHCP should satisfy the property. Following the idea of the property, we employ a method of perturbing kernel and a Fourier truncation method for high order numerical differentiation and a two-dimensional inverse heat conduction problem. Concluding previous analysis and discussion, we propose the idea of modifying kernel for ill-posed problems whose solutions have a common form in the frequency space. Based on the idea, we employ the perturbation method to study a non-standard inverse heat conduction problem, Cauchy problem for Laplace equation and backward heat conduction problem.For all these regularization methods, we discuss the stability, and give and prove the convergence estimate between the exact solution and its regularized approximation.In addition, we discuss the numerical implementation of all these methods: we expatiate on the skill of applying Fourier transform and finite difference. Moreover, we give a large number of numerical examples to test various properties of the proposed regularization methods. These tests show that our methods are effective and numerically feasible.