Apple already has the CoreML framework anyone can use to do on device processing. They even have tools to help convert things into the format the iPhone uses. As others have pointed out you can already find apps with public models on the Apple App Store.
Or is there a common API for llm constitution (or however it is called)?