On envelopes of holomorphy of domains covered by Levi-flat hats and the reflection principle  [ Sur l’enveloppe d’holomorphie de domaines recouverts par des "chapeaux" Levi-plats et le principe de réflexion ]
Annales de l'Institut Fourier, Tome 52 (2002) no. 5, pp. 1443-1523.

Dans cet article, nous associons les techniques du principe de réflexion de Lewy-Pinchuk avec celles du principe de continuité de Behnke-Sommer. Après avoir prolongé holomorphiquement une fonction dite "de réflexion" à une congruence de sous-variétés de Segre, nous sommes conduits à l’étude de l’enveloppe d’holomorphie d’un domaine recouvert d’un "chapeau" Levi-plat lisse. D’après notre résultat principal, tout CR-difféomorphisme h:MM de classe 𝒞 entre deux hypersurfaces analytiques réelles globalement minimales de n (n2) est analytique réel en tout point de M si M est holomorphiquement non-dégénérée. Plus généralement, nous établissons que la fonction de réflexion h associée à un tel difféomorphisme CR de classe 𝒞 entre deux hypersurfaces analytiques réelles globalement minimales se prolonge toujours holomorphiquement à un voisinage du graphe de h ¯ dans M×M ¯ , sans aucune condition de non-dégénérescence sur M . Cet énoncé fournit une nouvelle version du principe de réflexion de Schwarz en plusieurs variables complexes. Enfin, nous démontrons que toute application h:MM de classe 𝒞 et CR entre deux hypersurfaces analytiques réelles ne contenant pas de courbes holomorphes est analytique réelle en tout point de M, sans aucune condition de rang sur h.

In the present paper, we associate the techniques of the Lewy-Pinchuk reflection principle with the Behnke-Sommer continuity principle. Extending a so-called reflection function to a parameterized congruence of Segre varieties, we are led to studying the envelope of holomorphy of a certain domain covered by a smooth Levi-flat "hat". In our main theorem, we show that every 𝒞 -smooth CR diffeomorphism h:MM between two globally minimal real analytic hypersurfaces in n (n2) is real analytic at every point of M if M is holomorphically nondegenerate. More generally, we establish that the reflection function h associated to such a 𝒞 -smooth CR diffeomorphism between two globally minimal hypersurfaces in n (n1) always extends holomorphically to a neighborhood of the graph of h ¯ in M×M ¯ , without any nondegeneracy condition on M . This gives a new version of the Schwarz symmetry principle to several complex variables. Finally, we show that every 𝒞 - smooth CR mapping h:MM between two real analytic hypersurfaces containing no complex curves is real analytic at every point of M, without any rank condition on h.

DOI : https://doi.org/10.5802/aif.1922
Classification : 32V25,  32V40,  32V10,  32D10,  32D20
Mots clés: principe de réflexion, principe de continuité, difféomorphisme CR, non dégénérescence holomorphe, minimalité globale au sens de Trépreau-Tumanov, fonction de réflexion, enveloppes d’holomorphie
@article{AIF_2002__52_5_1443_0,
     author = {Merker, Jo\"el},
     title = {On envelopes of holomorphy of domains covered by Levi-flat hats and the reflection principle},
     journal = {Annales de l'Institut Fourier},
     pages = {1443--1523},
     publisher = {Association des Annales de l'institut Fourier},
     volume = {52},
     number = {5},
     year = {2002},
     doi = {10.5802/aif.1922},
     mrnumber = {1935554},
     zbl = {1031.32028},
     language = {en},
     url = {aif.centre-mersenne.org/item/AIF_2002__52_5_1443_0/}
}
Merker, Joël. On envelopes of holomorphy of domains covered by Levi-flat hats and the reflection principle. Annales de l'Institut Fourier, Tome 52 (2002) no. 5, pp. 1443-1523. doi : 10.5802/aif.1922. https://aif.centre-mersenne.org/item/AIF_2002__52_5_1443_0/

R.A. Ajrapetyan Extension of CR-functions from piecewise smooth CR manifolds, Math. USSR Sb. (English transl.), Tome 62 (1989) no. 1, pp. 111-120 | Article | Zbl 0635.32012

A.E. Tumanov Extending CR functions on a manifold of finite type over a wedge, Math. USSR Sb., Tome 64 (1989) no. 1, pp. 129-140 | Article | MR 912414 | Zbl 0663.32015

[A] R.A. Ajrapetyan Extension of CR-functions from piecewise smooth CR manifolds, Mat. Sb., Tome 134 (1987) no. 176-1, pp. 108-118 | Article | MR 232018 | Zbl 0172.05301

[Ar] M. Artin On the solutions of analytic equations, Invent. Math., Tome 5 (1968), pp. 277-291 | Article | MR 512898 | Zbl 0401.32007

[BeFo] E. Bedford; J.E. Fornaess Local extension of CR functions from weakly pseudoconvex boundaries, Michigan Math. J., Tome 25 (1978) no. 3, pp. 259-262 | Article | Zbl 0681.32010

[BePi] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild; E. Bedford; S. Pinchuk Analytic continuation of biholomorphic maps, Michigan Math. J., Tome 28 (1987) no. 3, pp. 405-408 | Article | MR 1440933 | Zbl 0890.32005

[BER1] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Algebraicity of holomorphic mappings between real algebraic sets in $\C^n$, Acta Math., Tome 177 (1996) no. 2, pp. 225-273 | Article | Zbl 0944.32040

[BER2] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Real submanifolds in complex space and their mappings (Princeton Mathematical Series) Tome 47 (1999), pp. 404 p. | Article | MR 1775734 | Zbl 0958.32033

[BER3] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Convergence and finite determinacy of formal CR mappings, J. Amer. Math. Soc., Tome 13 (2000) no. 4, pp. 697-723 | Article | MR 1754643 | Zbl 0955.32027

[BER4] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Local geometric properties of real submanifolds in complex space, Bull. Amer. Math. Soc., Tome 37 (2000) no. 3, pp. 309-336 | Article | MR 1389959 | Zbl 0855.32009

[BHR] M.S. Baouendi; X. Huang; L.P. Rothschild Regularity of CR mappings between algebraic hypersurfaces, Invent. Math., Tome 125 (1996) no. 1, pp. 13-36 | Article | MR 808223 | Zbl 0583.32021

[BJT] M.S. Baouendi; H. Jacobowitz; F. Treves On the analyticity of CR mappings, Ann. of Math., Tome 122 (1985) no. 2, pp. 365-400 | Article | | MR 952280 | Zbl 0653.32020

[BR1] M.S. Baouendi; L.P. Rothschild Germs of CR maps between real analytic hypersurfaces, Invent. Math., Tome 93 (1988) no. 3, pp. 481-500 | Article | MR 1037411 | Zbl 0702.32014

[BR2] M.S. Baouendi; L.P. Rothschild Geometric properties of mappings between hypersurfaces in complex space, J. Differential Geom., Tome 31 (1990) no. 2, pp. 473-499 | Article | | MR 1055709 | Zbl 0712.32009

[BR3] M.S. Baouendi; L.P. Rothschild Cauchy-Riemann functions on manifolds of higher codimension in complex space, Invent. Math., Tome 101 (1990) no. 1, pp. 45-56 | Article | MR 1317227 | Zbl 0869.14025

[BR4] M.S. Baouendi; L.P. Rothschild Mappings of real algebraic hypersurfaces, J. Amer. Math. Soc., Tome 8 (1995) no. 4, pp. 997-1015 | Article | MR 607899 | Zbl 0491.35036

[BT1] M.S. Baouendi; F. Treves A property of the functions and distributions annihilated by a locally integrable system of complex vector fields, Ann. of Math., Tome 113 (1981) no. 2, pp. 387-421 | Article | MR 744289 | Zbl 0564.32011

[BT2] M.S. Baouendi; F. Treves About the holomorphic extension of CR functions on real hypersurfaces in complex space, Duke Math. J., Tome 51 (1984) no. 1, pp. 77-107 | Article | MR 1666972 | Zbl 0926.32044

[CMS] B. Coupet; F. Meylan; A. Sukhov Holomorphic maps of algebraic CR manifolds, Internat. Math. Res. Notices, Tome 1 (1999), pp. 1-29 | Article | MR 1800211 | Zbl 0972.32008

[CPS2] B. Coupet; S. Pinchuk; A. Sukhov Analyticité des applications CR, C.R. Acad. Sci. Paris, Sér. I Math., Tome 329 (1999) no. 6, pp. 489-494 | Article | MR 1715137 | Zbl 0949.32018

[CPS1] B. Coupet; S. Pinchuk; A. Sukhov On partial analyticity of CR mappings, Math. Z., Tome 235 (2000), pp. 541-557 | Article | MR 1224231 | Zbl 0854.32001

[D'A] J.P. D'Angelo Several Complex Variables and the Geometry of Real Hypersurfaces, Studies in Advanced Mathematics, CRC Press, Boca Raton, FL, 1993 | MR 1834056 | Zbl 1066.14509

[Da1] S. Damour Sur l'algébricité des applications holomorphes, C. R. Acad. Sci. Paris, Sér. I Math., Tome 332 (2001) no. 6, pp. 491-496 | Article | MR 1872758 | Zbl 0997.32033

[Da2] S. Damour On the analyticity of smooth CR mappings between real analytic CR manifolds, Michigan Math. J., Tome 49 (2001) no. 3, pp. 583-603 | Article | MR 774535 | Zbl 0538.32009

[De] M. Derridj Le principe de réflexion en des points de faible pseudoconvexité pour des applications holomorphes propres, Invent. Math., Tome 79 (1985) no. 1, pp. 197-215 | Article | MR 477153 | Zbl 0378.32014

[DF1] K. Diederich; J.E. Fornaess Pseudoconvex domains with real analytic boundary (1978), pp. 371-384 | Article | MR 553344 | Zbl 0418.32018

[DF2] K. Diederich; J.E. Fornaess Biholomorphic maps between certain real analytic domains in ${\Bbb C}^2$, Math. Ann., Tome 245 (1979) no. 3, pp. 255-272 | Article | Zbl 0504.32011

[DF3] K. Diederich; J.E. Fornaess; J.E. Fornaess, ed. Biholomorphic mappings between two-dimensional Hartogs domains with real analytic boundaries, Recent Developments in Several Complex Variables (Ann. Math. Studies) Tome 100 (1981), pp. 133-150 | Article | MR 970228 | Zbl 0661.32025

[DF4] K. Diederich; J.E. Fornaess Proper holomorphic mappings between real-analytic pseudoconvex domains in $\C^n$., Math. Ann., Tome 282 (1988) no. 4, pp. 681-700 | Article | MR 1305994 | Zbl 0864.32014

[DFY] K. Diederich; J.E. Fornaess; Z. Ye Biholomorphisms in dimension 2, J. Geom. Anal., Tome 4 (1994) no. 4, pp. 539-552 | MR 1386762 | Zbl 0857.32015

[DP1] K. Diederich; S. Pinchuk Proper holomorphic maps in dimension 2 extend, Indiana Univ. Math. J., Tome 44 (1995) no. 4, pp. 1089-1126 | Zbl 0914.32008

[DP2] K. Diederich; S. Pinchuk Reflection principle in higher dimensions, ICM, Berlin, Doc. Math. (Proceedings of the International Congress of Mathematicians) Tome Extra Vol. II (1998), pp. 703-712 | Article | MR 596117 | Zbl 0451.32008

[DW] K. Diederich; S.M. Webster A reflection principle for degenerate real hypersurfaces, Duke Math. J., Tome 47 (1980) no. 4, pp. 835-843 | Article | MR 969413 | Zbl 0633.32017

[F] F. Forstneric Extending proper holomorphic mappings of positive codimension, Invent. Math., Tome 95 (1989) no. 1, pp. 31-62 | Article | MR 707348 | Zbl 0517.32007

[Ha] C.K. Han Analyticity of CR equivalences between some real analytic hypersurfaces in $\C^n$ with degenerate Levi-forms, Invent. Math., Tome 73 (1983) no. 1, pp. 51-69 | Article | MR 698000 | Zbl 0494.32004

[HaTr] N. Hanges; F. Treves Propagation of holomorphic extendability of CR functions, Math. Ann., Tome 263 (1983) no. 2, pp. 157-177 | Article | Zbl 0967.32030

[HMM] X. Huang; J. Merker; F. Meylan Mappings between degenerate real analytic hypersurfaces in ${\Bbb C}^n$, Analysis, geometry, number theory: the mathematics of Leon Ehrenpreis (Philadelphia, PA, 1998) (Contemp. Math.) Tome 251 (2000), pp. 321-338 | MR 1421212 | Zbl 0886.32010

[Hu] X. Huang Schwarz reflection principle in complex spaces of dimension two, Comm. Partial Differential Equations, Tome 21 (1996) no. 11-12, pp. 1781-1828 | MR 1601405 | Zbl 0917.32007

[J] B. Jöricke Deformation of CR-manifolds, minimal points and CR-manifolds with the microlocal analytic extension property, J. Geom. Anal., Tome 6 (1996) no. 4, pp. 555-611

[L] H. Lewy On the boundary behaviour of holomorphic mappings, Contrib. Centro Linceo Inter. Sc. Mat. e Loro Appl., Accad. Naz. Lincei, Tome 35 (1977), pp. 1-8 | Zbl 0177.17902

[Ma] B. Malgrange Ideals of Differentiable Functions (Tata Institute of Fundamental Research Studies in Mathematics, Bombay) Tome No. 3 (1967), pp. 106 p. | Article | MR 1443041 | Zbl 0885.32017

[MaMe] H. Maire; F. Meylan Extension of smooth CR mappings between non-essentially finite hypersurfaces in $\C^3$, Ark. Math., Tome 35 (1997) no. 1, pp. 185-199 | Article | MR 1289578 | Zbl 0815.32007

[Me1] J. Merker Global minimality of generic manifolds and holomorphic extendibility of CR functions, Internat. Math. Res. Notices (1994) no. 8, pp. approx. 14 p. (electronic) 329-342 | MR 1426732 | Zbl 0880.32009

[Me2] J. Merker On removable singularities for CR functions in higher codimension, Internat. Math. Res. Notices (1997) no. 1, pp. approx. 36 p. (electronic) 21-56

[Me3] J. Merker On the Schwarz symmetry principle in three-dimensional complex euclidean space (1997) (Prépublication Ecole Normale Supérieure, 25, 62 p.)

[Me4] J. Merker Vector field construction of Segre sets (1998, augmented in 2000) (Preprint 1998, \tt arXiv.org/abs/math.CV/9901010) | Numdam | MR 1894150 | Zbl 0998.32019

[Me5] J. Merker On the partial algebraicity of holomorphic mappings between two real algebraic sets in the complex euclidean spaces of different dimensions, Bull. Soc. Math. France, Tome 129 (2001) no. 4, pp. 547-591 | Article | MR 1854009 | Zbl 1002.32029

[Me7] J. Merker Convergence of formal invertible CR mappings between minimal holomorphically nondegenerate real analytic hypersurfaces, Int. J. Math. Math. Sci., Tome 26 (2001) no. 5, pp. 281-302 | Article | MR 1854009 | Zbl 1002.32029

[Me8] J. Merker Étude de la régularité analytique de l'application de symétrie CR formelle, C. R. Acad. Sci. Paris, Sér. I Math., Tome 333 (2001) no. 3, pp. 165-168 | MR 1851618 | Zbl 0994.32028

[Me6] J. Merker Étude de la régularité analytique de l'application de symétrie CR formelle (June 2000) (Preprint, \tt arXiv.org/math/abs/0005290) | MR 1375349 | Zbl 0857.32007

[Mey] F. Meylan The reflection principle in complex space, Indiana Univ. Math. J., Tome 44 (1995) no. 3, pp. 783-796 | Article | MR 1696763 | Zbl 0930.32020

[Mi3] N. Mir On the convergence of formal mappings (To appear in Comm. Anal. Geom.) | Article | Numdam | MR 1656006 | Zbl 0914.32009

[Mi2] N. Mir Germs of holomorphic mappings between real algebraic hypersurfaces, Ann. Inst. Fourier, Tome 48 (1998) no. 3, pp. 1025-1043 | Article | MR 1894140 | Zbl 1008.32022

[Mi1] N. Mir An algebraic characterization of holomorphic nondegeneracy for real algebraic hypersurfaces and its application to CR mappings, Math. Z., Tome 231 (1999) no. 1, pp. 189-202 | Article | MR 1764327 | Zbl 0964.32013

[Mi4] N. Mir Formal biholomorphic maps of real analytic hypersurfaces, Math. Res. Letters, Tome 7 (2000), p. 2-3, 343-359 | MR 1742868 | Zbl 1025.32029

[MM1] J. Merker; F. Meylan Extension de germes de difféomorphismes CR pour une classe d'hypersurfaces analytiques réelles non essentiellement finies dans $\C^3$, Complex variables Theory Appl., Tome 40 (1999) no. 1, pp. 19-34 | Article | MR 1476379 | Zbl 0919.32011

[MM2] J. Merker; F. Meylan On the Schwarz symmetry principle in a model case, Proc. Amer. Math. Soc., Tome 127 (1999) no. 4, pp. 1097-1102 | Article | MR 1736977 | Zbl 0935.32010

[MP2] J. Merker; E. Porten On wedge extendability of CR-meromorphic functions (To appear in Math. Z) | MR 1938701 | Zbl 1026.32020

[MP1] J. Merker; E. Porten On removable singularities for integrable CR functions, Indiana Univ. Math. J., Tome 48 (1999) no. 3, pp. 805-856 | Article | MR 1396991 | Zbl 0890.32004

[N] T.S. Neelon On solutions of real analytic equations, Proc. Amer. Math. Soc., Tome 125 (1997) no. 3, pp. 2531-2535 | Article | MR 350065 | Zbl 0292.32002

[P1] S. Pinchuk A boundary uniqueness theorem for holomorphic functions of several complex variables, Mat. Zametki, Tome 15 (1974), pp. 205-212 | Zbl 0303.32016

[P2] S. Pinchuk On proper holomorphic mappings of strictly pseudoconvex domains (Russian), Sibirsk. Mat. Z., Tome 15 (1974), pp. 909-917 | MR 393562 | Zbl 0366.32010

[P3] S. Pinchuk On the analytic continuation of holomorphic mappings (Russian), Mat. Sb. (N.S.), Tome 98(140)-3(11) (1975), p. 375-392, 416-435, 495-496 | MR 496595 | Zbl 0438.32009

[P4] S. Pinchuk Holomorphic mappings of real-analytic hypersurfaces (Russian), Mat. Sb. (N.S.), Tome 105(147) (1978) no. 4, p. 574-593, 640

[Po] E. Porten Habilitationsschrift (In preparation) | Article | MR 1838785 | Zbl 0981.32024

[PV] S. Pinchuk; K. Verma Analytic sets and the boundary regularity of CR mappings, Proc. Amer. Math. Soc., Tome 129 (2001) no. 9, pp. 2623-2632 | Article | MR 725396 | Zbl 0568.32011

[R] C. Rea Prolongement holomorphe des fonctions CR, conditions suffisantes, C. R. Acad. Sci. Paris, Sér I Math., Tome 297 (1983) no. 3, pp. 163-165 | Article | MR 1755261 | Zbl 0966.32007

[Sha] R. Shafikov Analytic continuation of germs of holomorphic mappings between real hypesurfaces in $\C^n$, Michigan Math. J., Tome 47 (2000) no. 1, pp. 133-149 | Article | MR 1325920 | Zbl 0851.32017

[SS] R. Sharipov; A. Sukhov On CR mappings between algebraic Cauchy-Riemann manifolds and separate algebraicity for holomorphic functions, Trans. Amer. Math. Soc., Tome 348 (1996) no. 2, pp. 767-780 | Article | MR 1314461 | Zbl 0826.32013

[St1] N. Stanton Infinitesimal CR automorphisms of rigid hypersurfaces of the space of n complex variables, Amer. J. Math., Tome 117 (1995) no. 1, pp. 141-167 | Article | MR 1375306 | Zbl 0849.32012

[St2] N. Stanton Infinitesimal CR automorphisms of real hypersurfaces, Amer. J. Math., Tome 118 (1996) no. 1, pp. 209-233 | Article | MR 1218705 | Zbl 0848.32016

[Su1] A. Sukhov On the mapping problem for quadric Cauchy-Riemann manifolds, Indiana Univ. Math. J., Tome 42 (1993) no. 1, pp. 27-36 | Article | MR 1260615 | Zbl 0821.32015

[Su2] A. Sukhov On CR mappings of real quadric manifolds, Michigan Math. J., Tome 41 (1994) no. 1, pp. 143-150 | Article | MR 827369 | Zbl 0586.32016

[Tr1] J.-M. Trépreau Sur le prolongement holomorphe des fonctions CR définies sur une hypersurface réelle de classe ${\cal C}^2$ dans $\C^n$, Invent. Math., Tome 83 (1986), pp. 583-592 | Article | Numdam | MR 1090408 | Zbl 0742.58053

[Tr2] J.-M. Trépreau Sur la propagation des singularités dans les variétés CR, Bull. Soc. Math. Fr., Tome 118 (1990) no. 4, pp. 403-450 | Zbl 0851.32021

[Tr3] J.-M. Trépreau Holomorphic extension of CR functions: a survey, Partial differential equations and mathematical physics (Copenhagen, 1995; Lund 1995) (Progr. Nonlinear Differential Equations Appl.) Tome 21 (1996), pp. 333-355 | MR 945904 | Zbl 0692.58005

[Tu1] A.E. Tumanov Extending CR functions on a manifold of finite type over a wedge (Russian), Mat. Sb. (N.S.), Tome 136(178) (1988) no. 1, pp. 128-139 | Article | MR 945904 | Zbl 0692.58005

[Tu2] A.E. Tumanov Connections and propagation of analyticity for CR functions, Duke Math. J., Tome 73 (1994) no. 1, pp. 1-24 | Article | MR 1257276 | Zbl 0801.32005

[Tu3] A.E. Tumanov On the propagation of extendibility of CR functions, Complex analysis and geometry (Trento, 1993) (Lecture Notes in Pure and Appl. Math.) Tome 173 (1996), pp. 479-498 | Zbl 0849.32013

[V] K. Verma Boundary regularity of correspondences in $\C^2$, Math. Z., Tome 231 (1999) no. 2, pp. 253-299 | Article | MR 1703349 | Zbl 0939.32014

[W1] S.M. Webster On the mapping problem for algebraic real hypersurfaces, Invent. Math., Tome 43 (1977) no. 1, pp. 53-68 | Article | MR 463482 | Zbl 0348.32005

[W2] S.M. Webster On the reflection principle in several complex variables, Proc. Amer. Math. Soc., Tome 71 (1978) no. 1, pp. 26-28 | Article | MR 477138 | Zbl 0626.32019

[W3] S.M. Webster Holomorphic mappings of domains with generic corners, Proc. Amer. Math. Soc., Tome 86 (1982) no. 2, pp. 236-240 | Article | MR 667281 | Zbl 0505.32014