First, they can be differential forms, not only functions. Second, there's an important note that we don't look only at things over C. For example, specifically in the context of Fermat's Last Theorem, we need Hida's theory of p-adic families of modular forms. Much of the arithmetic of modular forms comes from the modular curves being algebraic and (almost) defined over the integers.
The above definition (analytic function on a moduli space of elliptic curve) actually extends in a natural way. I haven’t known what modular forms were before the parent comment, but I know algebraic geometry, and so it is natural for me to extend above definition for cases you mention.
If modular forms are (global?) sections of the structural sheaf of the moduli space of elliptic curves, the differential forms view will just be the standard construction of sheaf of 1-differentials. Similarly, since elliptic curves are easily defined over arithmetic fields, arithmetic modular forms will just be same thing, but over C_p or something like that.
I actually might be totally off in the above, but I doubt I am: that’s the power of Grothendieck approach, where everything just falls into its natural place in the framework.
This definitely fits with Grothendieck's philosophy: he basically ignored all work in this area, implicitly claiming it was trivial, while some of his closest friends and most famous student made huge strides with actual hard work - not quite things falling into place. In fact, the paper most famously proving the Weil conjectures has as an explicit target the coefficients of a modular form, uses an inspiration from automorphic forms theory, and is infamously Grothendieck's greatest disappointment.
There is rich structure in this area of maths that goes well beyond just sections of some sheaf, or at least this is what Serre, Deligne, Langlands, Mazur, Katz, Hida, Taylor, Wiles and many others seem to think.
Oh, I did not meant to imply that the framework necessarily makes it so that the results open like a softened, rubbed nut, as Grothendieck said; I don't quite agree with that. For me, the benefit is rather in building a mental framework, which facilitates understanding, and putting seemingly disparate things into one coherent whole. The actual hard thinking and insights are still necessary, it ain't no royal road.
First, they can be differential forms, not only functions. Second, there's an important note that we don't look only at things over C. For example, specifically in the context of Fermat's Last Theorem, we need Hida's theory of p-adic families of modular forms. Much of the arithmetic of modular forms comes from the modular curves being algebraic and (almost) defined over the integers.