Isn't this true for any CGI program? I know that PHP is typically run via mod_php, but what you have described is true for any CGI program.
It is somewhat expensive to startup a new interpreter process per http connection. This can be alleviated by using FastCGI, but then the interpreters are long lived and are no longer completely independent.
Yes, though PHP comes with a few goodies such as mainstream mod_php and FastCGI (which I prefer), and bytecode cache (APC). The implementation is pretty good compared to a ye olde CGI Perl script.
It is somewhat expensive to startup a new interpreter process per http connection. This can be alleviated by using FastCGI, but then the interpreters are long lived and are no longer completely independent.