A geometric approach to Catlin’s boundary systems
Annales de l'Institut Fourier, Volume 69 (2019) no. 6, p. 2635-2679

For a point p in a smooth real hypersurface M n , where the Levi form has the nontrivial kernel K p 10 0, we introduce an invariant cubic tensor

τp3:Tp×Kp10×Kp10¯(Tp/Hp),

which together with Ebenfelt’s 3rd order tensor, constitutes the full set of the 3rd order invariants of M at p.

Next, in addition, assume M n to be (weakly) pseudoconvex. Then τ p 3 must identically vanish. In this case we further define an invariant quartic tensor

τp4:Tp×Tp×Kp10×Kp10¯(Tp/Hp),

and for every q=0,,n-1, an invariant submodule sheaf 𝒮 10 (q) of (1,0) vector fields in terms of the Levi form, and an invariant ideal sheaf (q) of complex functions generated by certain components and derivatives of the Levi form, such that the set of points of Levi rank q is locally contained in real smooth submanifolds defined by real parts of the functions in (q), whose tangent spaces have explicit algebraic description in terms of the quartic tensor τ 4 .

Most recently, the constructions of τ 3 and τ 4 inspired Raich and Harrington [35] to discover new related invariants in the non-pseudoconvex case and connect them with the closed range property for the ¯ operator.

Finally, we relate the introduced invariants with D’Angelo’s finite type, Catlin’s multitype and Catlin’s boundary systems.

Soit p un point d’une hypersurface M n , réelle et lisse, en lequel la forme de Levi possède un noyau non trivial, noté K p 10 0. On introduit alors un tenseur d’ordre 3, invariant

τp3:Tp×Kp10×Kp10¯(Tp/Hp),

qui, avec le tenseur d’ordre 3 d’Ebenfelt, fournit l’ensemble de tous les tenseurs d’ordre 3 invariants de M au point p.

Si, de plus, on suppose que M est pseudo-convexe, alors τ p 3 est identiquement nul. On définit alors un tenseur d’ordre 4, invariant

τp4:Tp×Tp×Kp10×Kp10¯(Tp/Hp).

On introduit également pour q=0,...,n-1, un faisceau de sous-modules invariant, noté 𝒮 10 (q), constitué de champs de vecteurs de type (1,0), construits à partir de la forme de Levi, ainsi qu’un faisceau d’idéaux invariant, noté (q), constitué de fonctions complexes construites à partir de certaines composantes et dérivées de la forme de Levi. L’ensemble des points dont la forme de Levi est de rang q est alors contenu localement dans des sous-variétés réelles et lisses définies par des parties rélles de fonctions dans (q), et dont les espaces tangents ont des descriptions algébriques explicites en termes du tenseur τ 4 .

Tout récemment, la construction de τ 3 et τ 4 a inspiré Raich et Harrington [35] qui ont découvert de nouveaux invariants associés à ces derniers dans le cas où M n’est pas pseudo-convexe et les ont reliés à la « closed range property » pour l’opérateur ¯.

Enfin, on fait le lien entre ces invariants définis plus haut et le type fini au sens de D’Angelo, le multitype au sens de Catlin et les « boundary systems » de Catlin.

Received : 2018-01-22
Accepted : 2018-07-12
Published online : 2019-10-29
DOI : https://doi.org/10.5802/aif.3304
Classification:  32T25,  32T27,  32V05,  32V15,  32V35,  32W05,  32S60,  58K50
Keywords: Catlin multitype, subelliptic estimates, boundary systems, Levi form, pseudoconvexity, real hypersurfaces, invariant tensors, ideal sheaves
@article{AIF_2019__69_6_2635_0,
     author = {Zaitsev, Dmitri},
     title = {A geometric approach to Catlin's boundary systems},
     journal = {Annales de l'Institut Fourier},
     publisher = {Association des Annales de l'institut Fourier},
     volume = {69},
     number = {6},
     year = {2019},
     pages = {2635-2679},
     doi = {10.5802/aif.3304},
     language = {en},
     url = {https://aif.centre-mersenne.org/item/AIF_2019__69_6_2635_0}
}
Zaitsev, Dmitri. A geometric approach to Catlin’s boundary systems. Annales de l'Institut Fourier, Volume 69 (2019) no. 6, pp. 2635-2679. doi : 10.5802/aif.3304. https://aif.centre-mersenne.org/item/AIF_2019__69_6_2635_0/

[1] Baracco, Luca; Khanh, Tran Vu; Pinton, Stefano; Zampieri, Giuseppe Hölder regularity of the solution to the complex Monge-Ampère equation with L p density, Calc. Var. Partial Differ. Equ., Tome 55 (2016) no. 4, 74, 8 pages | Zbl 1361.32045

[2] Bharali, Gautam; Stensønes, Berit Plurisubharmonic polynomials and bumping, Math. Z., Tome 261 (2009) no. 1, pp. 39-63 | Article | MR 2452636 | Zbl 1185.32025

[3] Biard, Séverine; Straube, Emil J. L 2 -Sobolev theory for the complex Green operator, Int. J. Math., Tome 28 (2017) no. 9, 1740006, 31 pages | MR 3690415 | Zbl 1380.32036

[4] Bierstone, Edward; Milman, Pierre D. Ideals of holomorphic functions with C boundary values on a pseudoconvex domain, Trans. Am. Math. Soc., Tome 304 (1987) no. 1, pp. 323-342 | MR 906818 | Zbl 0631.32015

[5] Boas, Harold P. The Szegö projection: Sobolev estimates in regular domains, Trans. Am. Math. Soc., Tome 300 (1987) no. 1, pp. 109-132 | Zbl 0622.32006

[6] Boas, Harold P.; Straube, Emil J.; Yu, Ji-Ye Boundary limits of the Bergman kernel and metric, Mich. Math. J., Tome 42 (1995) no. 3, pp. 449-461 | MR 1357618 | Zbl 0853.32028

[7] Catlin, David W. Global regularity of the ¯-Neumann problem, Complex analysis of several variables (Madison,1982), American Mathematical Society (Proceedings of Symposia in Pure Mathematics) Tome 41 (1982), pp. 39-49 | Article | MR 740870 | Zbl 0578.32031

[8] Catlin, David W. Boundary invariants of pseudoconvex domains, Ann. Math., Tome 120 (1984), pp. 529-586 | Article | MR 769163 | Zbl 0583.32048

[9] Catlin, David W. Subelliptic estimates for the ¯-Neumann problem on pseudoconvex domains, Ann. Math., Tome 126 (1987), pp. 131-191 | Article | MR 898054 | Zbl 0627.32013

[10] Catlin, David W. Estimates of invariant metrics on pseudoconvex domains of dimension two, Math. Z., Tome 200 (1989) no. 3, pp. 429-466 | Article | MR 978601 | Zbl 0661.32030

[11] Catlin, David W.; D’Angelo, John P. Subelliptic estimates, Complex analysis. Several complex variables and connections with PDE theory and geometry, Birkhäuser/Springer (Trends in Mathematics) (2010), pp. 75-94 | Zbl 1202.32027

[12] Çelik, Mehmet; Şahutoğlu, Sönmez On compactness of the ¯-Neumann problem and Hankel operators, Proc. Am. Math. Soc., Tome 140 (2012) no. 1, pp. 153-159 | Article | MR 2833527 | Zbl 1251.32032

[13] Charpentier, Philippe; Dupain, Yves Estimates for the Bergman and Szegö projections for pseudoconvex domains of finite type with locally diagonalizable Levi form, Publ. Mat., Barc., Tome 50 (2006) no. 2, pp. 413-446 | Article | Zbl 1120.32002

[14] Charpentier, Philippe; Dupain, Yves Geometry of pseudo-convex domains of finite type with locally diagonalizable Levi form and Bergman kernel, J. Math. Pures Appl., Tome 85 (2006) no. 1, pp. 71-118 | Article | MR 2200592 | Zbl 1092.32017

[15] Charpentier, Philippe; Dupain, Yves Extremal bases, geometrically separated domains and applications, Algebra Anal., Tome 26 (2014) no. 1, pp. 196-269 (translation in St. Petersbg. Math. J. 26 (2015), no. 1, p. 139-191) | MR 3234809 | Zbl 1343.32023

[16] Chen, Bo-Yong; Fu, Siqi Comparison of the Bergman and Szegö kernels, Adv. Math., Tome 228 (2011) no. 4, pp. 2366-2384 | Article | Zbl 1229.32008

[17] Cho, Sanghyun A lower bound on the Kobayashi metric near a point of finite type in n , J. Geom. Anal., Tome 2 (1992) no. 4, pp. 317-325 | MR 1170478 | Zbl 0756.32015

[18] Cho, Sanghyun Boundary behavior of the Bergman kernel function on some pseudoconvex domains in n , Trans. Am. Math. Soc., Tome 345 (1994) no. 2, pp. 803-817 | MR 1254189 | Zbl 0813.32023

[19] Cho, Sanghyun Estimates of invariant metrics on pseudoconvex domains with comparable Levi form, J. Math. Kyoto Univ., Tome 42 (2002) no. 2, pp. 337-349 | MR 1966842 | Zbl 1036.32013

[20] Christ, Michael; Fu, Siqi Compactness in the ¯-Neumann problem, magnetic Schrödinger operators, and the Aharonov-Bohm effect, Adv. Math., Tome 197 (2005) no. 1, pp. 1-40 | Article | Zbl 1098.32020

[21] D’Angelo, John P. Subelliptic estimates and failure of semicontinuity for orders of contact, Duke Math. J., Tome 47 (1980) no. 4, pp. 955-957 | MR 596122 | Zbl 0455.32010

[22] D’Angelo, John P. Real hypersurfaces, orders of contact, and applications, Ann. Math., Tome 115 (1982), pp. 615-637 | Article | MR 657241 | Zbl 0488.32008

[23] D’Angelo, John P. Several complex variables and the geometry of real hypersurfaces, CRC Press, Studies in Advanced Mathematics (1993) | Zbl 0854.32001

[24] Diederich, Klas; Herbort, Gregor Pseudoconvex domains of semiregular type, Contributions to complex analysis and analytic geometry, Vieweg & Sohn (Aspects of Mathematics) Tome E26 (1994), pp. 127-161 | Zbl 0845.32019

[25] Ebenfelt, Peter New invariant tensors in CR structures and a normal form for real hypersurfaces at a generic Levi degeneracy, J. Differ. Geom., Tome 50 (1998) no. 2, pp. 207-247 | Article | MR 1684982 | Zbl 0945.32020

[26] Fornæss, John E.; Mcneal, Jeffery D. A construction of peak functions on some finite type domains, Am. J. Math., Tome 116 (1994) no. 3, pp. 737-755 | Article | MR 1277453 | Zbl 0809.32005

[27] Fornæss, John E.; Sibony, Nessim Construction of P.S.H. functions on weakly pseudoconvex domains, Duke Math. J., Tome 58 (1989) no. 3, pp. 633-655 | MR 1016439 | Zbl 0679.32017

[28] Freeman, Michael Local biholomorphic straightening of real submanifolds, Ann. Math., Tome 106 (1977) no. 2, pp. 319-352 | Article | MR 463480 | Zbl 0372.32005

[29] Fu, Siqi; Jacobowitz, Howard The ¯-cohomology groups, holomorphic Morse inequalities, and finite type conditions, Pure Appl. Math. Q., Tome 6 (2010) no. 3, pp. 875-914 | MR 2677317 | Zbl 1214.32016

[30] Fu, Siqi; Straube, Emil J. Compactness in the ¯-Neumann problem, Complex analysis and geometry (Columbus, 1999), Walter de Gruyter (Ohio State University Mathematical Research Institute Publications) Tome 9 (2001), pp. 141-160 | MR 1912737 | Zbl 1011.32025

[31] Ha, Ly Kim; Khanh, Tran Vu Boundary regularity of the solution to the complex Monge-Ampère equation on pseudoconvex domains of infinite type, Math. Res. Lett., Tome 22 (2015) no. 2, pp. 467-484 | Zbl 1343.32030

[32] Harrington, Phillip S. Compact and subelliptic estimates for the ¯-Neumann operator on C 2 pseudoconvex domains, Math. Ann., Tome 337 (2007) no. 2, pp. 335-352 | Article | MR 2262787 | Zbl 1118.32027

[33] Harrington, Phillip S. Property (P) and Stein neighborhood bases on C 1 domains, Ill. J. Math., Tome 52 (2008) no. 1, pp. 145-151 | Article | MR 2507238 | Zbl 1175.32017

[34] Harrington, Phillip S. Global regularity for the ¯-Neumann operator and bounded plurisubharmonic exhaustion functions, Adv. Math., Tome 228 (2011) no. 4, pp. 2522-2551 | Article | MR 2836129 | Zbl 1328.32030

[35] Harrington, Phillip S.; Raich, Andrew Boundary invariants and the closed range property for ¯ (2018) (https://arxiv.org/abs/1805.05793) | Zbl 06817615

[36] Henkin, Gennadi M.; Iordan, Andrei Compactness of the Neumann operator for hyperconvex domains with non-smooth B-regular boundary, Math. Ann., Tome 307 (1997) no. 1, pp. 151-168 | Article | MR 1427681 | Zbl 0869.32009

[37] Herbig, Anne-Katrin A sufficient condition for subellipticity of the ¯-Neumann operator, J. Funct. Anal., Tome 242 (2007) no. 2, pp. 337-362 | Article | MR 2274813 | Zbl 1118.32028

[38] Herbort, Gregor On the Bergman metric on bounded pseudoconvex domains an approach without the Neumann operator, Int. J. Math., Tome 25 (2014) no. 3, 1450025, 13 pages | MR 3189782 | Zbl 1295.32021

[39] Kaup, Wilhelm; Zaitsev, Dmitri On local CR-transformation of Levi-degenerate group orbits in compact Hermitian symmetric spaces, J. Eur. Math. Soc., Tome 8 (2006) no. 3, pp. 465-490 | Article | MR 2250168 | Zbl 1118.32019

[40] Khanh, Tran Vu; Pinton, Stefano; Zampieri, Giuseppe Compactness estimates for b on a CR manifold, Proc. Am. Math. Soc., Tome 140 (2012) no. 9, pp. 3229-3236 | Article | Zbl 1300.32036

[41] Khanh, Tran Vu; Raich, Andrew Local regularity of the Bergman projection on a class of pseudoconvex domains of finite type (2014) (https://arxiv.org/abs/1406.6532)

[42] Khanh, Tran Vu; Zampieri, Giuseppe Necessary geometric and analytic conditions for general estimates in the ¯-Neumann problem, Invent. Math., Tome 188 (2012) no. 3, pp. 729-750 | Article | MR 2917182 | Zbl 1362.32025

[43] Kohn, Joseph J. Harmonic integrals on strongly pseudo-convex manifolds. I, Ann. Math., Tome 78 (1963), pp. 112-148 | Article | MR 153030 | Zbl 0161.09302

[44] Kohn, Joseph J. Harmonic integrals on strongly pseudo-convex manifolds. II, Ann. Math., Tome 79 (1964), pp. 450-472 | Article | MR 208200 | Zbl 0178.11305

[45] Kohn, Joseph J. Boundary behavior of ¯ on weakly pseudo-convex manifolds of dimension two, J. Differ. Geom., Tome 6 (1972), pp. 523-542 | Article | MR 322365 | Zbl 0256.35060

[46] Kohn, Joseph J. Subellipticity of the ¯-Neumann problem on pseudo-convex domains: sufficient conditions, Acta Math., Tome 142 (1979) no. 1-2, pp. 79-122 | Article | MR 512213 | Zbl 0395.35069

[47] Kolář, Martin The Catlin multitype and biholomorphic equivalence of models, Int. Math. Res. Not., Tome 2010 (2010) no. 18, pp. 3530-3548 | Article | MR 2725504 | Zbl 1207.32032

[48] Mcneal, Jeffery D. Lower bounds on the Bergman metric near a point of finite type, Ann. Math., Tome 136 (1992) no. 2, pp. 339-360 | Article | MR 1185122 | Zbl 0764.32006

[49] Mcneal, Jeffery D. Invariant metric estimates for ¯ on some pseudoconvex domains, Ark. Mat., Tome 39 (2001) no. 1, pp. 121-136 | Article | MR 1821085 | Zbl 1038.32004

[50] Mcneal, Jeffery D. A sufficient condition for compactness of the ¯-Neumann operator, J. Funct. Anal., Tome 195 (2002) no. 1, pp. 190-205 | Article | MR 1934357 | Zbl 1023.32029

[51] Mcneal, Jeffery D.; Varolin, Dror L 2 estimates for the ¯-operator, Bull. Math. Sci., Tome 5 (2015) no. 2, pp. 179-249 | Article | Zbl 1320.32045

[52] Nagel, Alexander; Rosay, Jean-Pierre; Stein, Elias M.; Wainger, Stephen Estimates for the Bergman and Szegö kernels in 2 , Ann. Math., Tome 129 (1989) no. 1, pp. 113-149 | Article | Zbl 0667.32016

[53] Nicoara, Andreea C. Direct proof of termination of the Kohn algorithm in the real-analytic case (2014) (https://arxiv.org/abs/1409.0963)

[54] Raich, Andrew Compactness of the complex Green operator on CR-manifolds of hypersurface type, Math. Ann., Tome 348 (2010) no. 1, pp. 81-117 | Article | MR 2657435 | Zbl 1238.32032

[55] Raich, Andrew; Straube, Emil J. Compactness of the complex Green operator, Math. Res. Lett., Tome 15 (2008) no. 4, pp. 761-778 | Article | MR 2424911 | Zbl 1157.32032

[56] Şahutoğlu, Sönmez Strong Stein neighbourhood bases, Complex Var. Elliptic Equ., Tome 57 (2012) no. 10, pp. 1073-1085 | Article | MR 2984006 | Zbl 1267.32036

[57] Siu, Yum-Tong Multiplier ideal sheaves in complex and algebraic geometry, Sci. China, Ser. A, Tome 48 (2005), pp. 1-31 | MR 2156488 | Zbl 1131.32010

[58] Siu, Yum-Tong Effective termination of Kohn’s algorithm for subelliptic multipliers, Pure Appl. Math. Q., Tome 6 (2010) no. 4, pp. 1169-1241 | MR 2742044 | Zbl 1235.32030

[59] Siu, Yum-Tong New procedure to generate multipliers in complex Neumann problem and effective Kohn algorithm, Sci. China, Math., Tome 60 (2017) no. 6, pp. 1101-1128 | MR 3647138 | Zbl 1402.32031

[60] Straube, Emil J. Plurisubharmonic functions and subellipticity of the ¯-Neumann problem on non-smooth domains, Math. Res. Lett., Tome 4 (1997) no. 4, pp. 459-467 | Article | MR 1470417 | Zbl 0887.32005

[61] Straube, Emil J. Aspects of the L 2 -Sobolev theory of the ¯-Neumann problem, Proceedings of the international congress of mathematicians (ICM). Volume II: Invited lectures, European Mathematical Society (2006), pp. 1453-1478 | MR 2275654 | Zbl 1106.32028

[62] Straube, Emil J. Lectures on the L 2 -Sobolev theory of the ¯-Neumann problem, European Mathematical Society, ESI Lectures in Mathematics and Physics (2010) | Zbl 1247.32003

[63] Straube, Emil J. The complex Green operator on CR-submanifolds of n of hypersurface type: compactness, Trans. Am. Math. Soc., Tome 364 (2012) no. 8, pp. 4107-4125 | Article | MR 2912447 | Zbl 1278.32027

[64] Webster, Sidney M. The holomorphic contact geometry of a real hypersurface, Modern methods in complex analysis, Princeton University Press (Annals of Mathematics Studies) Tome 137 (1995), pp. 327-342 | MR 1369146 | Zbl 0870.32008

[65] Yu, Ji-Ye Peak functions on weakly pseudoconvex domains, Indiana Univ. Math. J., Tome 43 (1994) no. 4, pp. 1271-1295 | MR 1322619 | Zbl 0828.32003