The key point is the trial. You can't hold a trial in a hostage situation. They could have held a trial for Anwar al-awlaki. Once decided by trial that he committed treason, military action could be justified.
The problem is how that trial would be conducted in absentia would have been a farce. The government would also be concerned about its classified evidence. Worse, if the trial led to acquittal, it would have legitimized his cause.
An acquittal is a scary prospect of bringing out other copycats. But more scary is executing someone without trial that would have been acquitted. But we'll never know. They should have held a trial.
Anwar al-Awlaki could have had a trial anytime he wanted. He simply had to go to the American Embassy and turn himself in. As things stood when he was killed, it was impossible to bring him back for a trial without an unacceptably high risks to the people in the armed forces who would have had to capture him. If you wanted some insight on the complexities of operating within Yemen, I suggest The Black Banners by Ali Soufan. The subject of the book is the events leading up to 9/11 and Soufan was one of the FBI's top agents and strong critic enhanced interrogation. He invested the USS Cole bombing in Yemen and he covers the difficulty of working in country quite well.
They would somehow have to serve papers telling him about the trial, and offer him safe passage to the trial. Then give him a reasonable deadline to show up. If he doesn't show, hand down a default judgement and proceed to sentencing...
Then you postpone the trial until they can be found. Trials in absentia are not allowed in the US and most of Europe. In countries where they are practiced, they're a source of international criticism.
The problem is how that trial would be conducted in absentia would have been a farce. The government would also be concerned about its classified evidence. Worse, if the trial led to acquittal, it would have legitimized his cause.
An acquittal is a scary prospect of bringing out other copycats. But more scary is executing someone without trial that would have been acquitted. But we'll never know. They should have held a trial.